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Why  GAO  Did  This  Study 

DOD’s  MAIS  programs  include 
systems  that  are  intended  to  help  the 
department  sustain  its  key  operations. 
The  National  Defense  Authorization 
Act  for  Fiscal  Year  2012  includes  a 
provision  for  GAO  to  select,  assess, 
and  report  on  the  department’s  MAIS 
programs  annually  through  March 
2018.  This  is  GAO’s  fifth  report  and  (1) 
describes  the  extent  to  which  selected 
MAIS  programs  have  changed  their 
planned  cost  and  schedule  estimates 
and  met  technical  performance  targets 
and  (2)  assesses  the  extent  to  which 
selected  MAIS  programs  have  used 
leading  IT  acquisition  practices, 
including  risk  management. 

GAO  selected  and  reviewed  cost, 
schedule,  and  performance  data  for  18 
of  DOD’s  MAIS  programs  that  were 
non-classified  and  had  an  acquisition 
performance  baseline.  In  addition, 

GAO  performed  an  in-depth  review  of 
5  of  the  programs,  comparing  selected 
IT  management  practices  used  by 
them  to  leading  practices  for 
requirements  and  risk  management 
and  systems  testing  and  integration. 
The  five  selected  programs  were  from 
at  least  two  military  services  and  had 
not  been  assessed  by  GAO  in  the  past 
year.  GAO  also  interviewed  relevant 
program  officials. 

What  GAO  Recommends 

GAO  recommends  that  DOD  improve 
the  management  of  specific  MAIS 
programs,  including  establishing 
procedures  for  defining  risk  thresholds, 
developing  an  overall  risk  mitigation 
plan,  and  filling  a  key  test  management 
position.  DOD  concurred  with  all  of  the 
recommendations. 


View  GAO-17-322.  For  more  information, 
contact  Carol  C.  Harris  at  (202)  512-4456  or 
harrisc@gao.gov. 
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What  GAO  Found 

Most  of  the  1 8  selected  Department  of  Defense  (DOD)  major  automated 
information  system  (MAIS)  programs  that  GAO  reviewed  had  experienced 
changes  in  their  planned  cost  and  schedule  estimates  and  half  of  the  programs 
had  met  their  technical  performance  targets.  Specifically,  16  programs 
experienced  changes  in  their  cost  estimates  ranging  from  a  39  percent  decrease 
($1 .47  billion)  to  a  469  percent  increase  ($1 .63  billion).  The  average  cost 
increase  was  $457.2  million  among  the  11  programs  reporting  an  increase. 
Fourteen  programs  experienced  schedule  delays,  which  ranged  from  2  months 
to  over  1 3  years.  Finally,  half  of  the  MAIS  programs  fully  met  all  of  their  technical 
performance  targets.  Of  the  remaining  nine  programs,  4  four  had  partially  met 
their  target  because  each  was  still  conducting  tests.  The  other  five  programs 
were  in  the  early  stages  of  system  development  and  had  not  begun  testing. 

In  addition,  for  the  five  MAIS  programs  GAO  selected  for  in-depth  review,  all  had 
either  fully  or  partially  applied  leading  practices  for  managing  requirements,  risks, 
and  for  conducting  systems  testing  and  integration. 

•  Managing  requirements.  Three  of  the  five  programs  had  fully  implemented 
the  practices  for  managing  requirements,  while  the  other  two  had  partially 
implemented  some  practices.  Leading  practices  in  this  area  include 
establishing  requirements  and  ensuring  traceability  between  requirements 
and  work  products. 

•  Managing  risks.  Three  of  the  five  programs  had  fully  implemented  the  risk 
management  practices,  while  two  had  partially  implemented  some  practices. 
An  effective  risk  management  process  identifies  potential  problems  before 
they  occur.  For  example,  one  Army  program  did  not  have  standard  operating 
procedures  for  defining  thresholds  or  bounds  to  manage  risk.  Unless  such 
procedures  are  defined,  the  program  will  not  have  the  tools  needed  to  define 
risk  management  activities,  including  whether  and  how  certain  risks  are 
prioritized.  Further,  programs  should  include  practices  to  identify  potential 
problems  so  that  risk-handling  activities  may  be  planned  and  invoked  across 
the  project  to  mitigate  the  potential  for  adverse  impacts.  However,  one  Air 
Force  program  did  not  develop  an  overall  risk  mitigation  plan  to  guide  the 
implementation  of  individual  risk  mitigation  activities.  Without  an  overall  risk 
plan  to  guide  individual  development  efforts,  those  efforts  cannot  be 
managed  cohesively. 

•  Testing  and  integration.  Four  of  the  five  programs  had  fully  implemented 
practices  for  systems  testing  and  integration.  Programs  should,  among  other 
activities,  establish  roles  and  responsibilities  to  manage  testing  and 
integration  activities,  including  a  chief  developmental  tester  to  oversee 
testing  activities.  However,  one  Air  Force  program  reported  difficulty  in  hiring 
a  qualified  individual  to  perform  these  duties.  Until  this  position  is  filled,  the 
program  may  not  effectively  manage  risks  and  verify  compliance  with  system 
acquisition  and  operational  requirements. 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


March  30,  2017 
Congressional  Committees 

The  Department  of  Defense  (DOD)  is  one  of  the  largest  and  most 
complex  organizations  in  the  world.  To  meet  its  mission  to  protect  the 
security  of  our  nation  and  to  deter  war,  it  relies  heavily  on  the  use  of 
information  technology  (IT)  to  support  our  warfighters.  In  this  regard, 
according  to  DOD’s  IT  investment  portfolio  for  fiscal  year  2016,  the 
department  spent  approximately  $31  billion  for  such  investments.1  Of  this 
amount,  approximately  $2.5  billion  was  spent  on  major  automated 
information  system  (MAIS)  programs,  which  include  systems  that  help  the 
department  sustain  its  key  operations  in  communications,  business,  and 
command  and  control,  and  provide  the  department  with  access  to 
information  to  organize,  plan,  direct,  and  monitor  mission  operations. 

A  DOD  IT  acquisition  investment  that  exceeds  one  of  the  following  cost 
thresholds  is  designated  as  a  MAIS  program:  (1)  program  costs  in  any 
single  year  exceed  $40  million,  (2)  total  program  acquisition  costs  exceed 
$165  million,  or  (3)  total  life-cycle  costs  exceed  $520  million  in  fiscal  year 
2014  constant  dollars.  The  Secretary  of  Defense  can  also  use  discretion 
to  designate  a  program  as  a  MAIS  if  it  does  not  meet  these  cost 
thresholds. 

The  National  Defense  Authorization  Act  for  Fiscal  Year  2012  includes  a 
provision  that  we  select,  assess,  and  report  on  DOD  MAIS  programs 
annually  through  March  201 8. 2  This  report  is  the  fifth  in  a  series  of  annual 
assessments.3  Our  objectives  for  this  review  were  to  (1 )  describe  the 


1  DOD’s  IT  investment  portfolio  identifies  all  of  its  IT  investments  and  associated  costs 
within  the  department  and  its  components. 

Effective  September  30,  2017,  the  statutory  major  automated  information  system 
provisions  are  repealed.  National  Defense  Authorization  Act  for  Fiscal  Year  2017,  Pub.  L. 
No.  114-328,  §  846  (Dec.  23,  2016). 

3GAO,  DOD  Major  Automated  Information  Systems:  Improvements  Can  Be  Made  in 
Reporting  Critical  Changes  and  Clarifying  Leadership  Responsibility,  GAO-16-336 
(Washington,  D.C.:  March  30,  2016) ;  Defense  Major  Automated  Information  Systems: 
Cost  and  Schedule  Commitments  Need  to  Be  Established  Earlier,  GAO- 15-282 
(Washington,  D.C.:  February  26,  2015) ;  Major  Automated  Information  Systems:  Selected 
Defense  Programs  Need  to  Implement  Key  Acquisition  Practices,  GAO-14-309 
(Washington,  D.C.:  March  27,  2014) ;  Major  Automated  Information  Systems:  Selected 
Defense  Programs  Need  to  Implement  Key  Acquisition  Practices,  GAO-13-31 1 
(Washington,  D.C.:  March  28,  2013). 
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extent  to  which  selected  MAIS  programs  had  changed  their  planned  cost 
and  schedule  estimates,  and  met  technical  performance  targets  and  (2) 
assess  the  extent  to  which  selected  MAIS  programs  have  used  leading  IT 
acquisition  practices,  including  requirements  and  risk  management,  and 
systems  testing  and  integration. 

To  address  the  first  objective,  we  used  DOD’s  official  list  of  35  MAIS 
programs,  as  of  May  1, 2016,  to  establish  a  basis  for  selecting  programs. 
We  selected  18  of  35  MAIS  programs  based  on  our  criteria  that  programs 
must  be  unclassified  and  have  a  first  acquisition  program  baseline  that 
could  be  used  as  a  reference  point  for  evaluating  cost,  schedule,  and 
technical  performance  characteristics. 

We  then  compared  each  program’s  cost  (in  then-year  dollars)  and 
schedule  estimates  established  in  the  first  acquisition  baseline  to  the 
latest  total  life-cycle  cost  and  schedule  estimates.4  In  addition,  to 
determine  whether  technical  performance  targets5  were  met,  we  reviewed 
each  program’s  initial  and  the  most  recent  baseline  targets.  We 
summarized  the  results  of  our  analyses  for  each  individual  MAIS 
program. 

To  address  the  second  objective,  we  selected  5  programs  for  a  more  in- 
depth  review  from  among  those  used  by  at  least  two  military  services  and 
that  had  not  been  included  in  a  GAO  assessment  in  the  past  year  to  focus 
on  programs  that  have  not  been  recently  evaluated.  Using  these  criteria, 
we  selected  the  5  programs  listed  in  table  1 . 


4Then-year  dollars  are  those  dollars  that  include  the  effects  of  inflation  or  escalation 
and/or  reflect  the  price  levels  expected  to  prevail  during  the  year  at  issue.  The  first 
acquisition  program  baseline  is  established  after  the  program  has  assessed  the  viability  of 
various  technologies  and  refined  user  requirements  to  identify  the  most  appropriate 
technology  solution  that  demonstrates  that  it  can  meet  users’  needs.  The  Defense 
Acquisition  Guidebook  (which  complements  and  further  explains  DOD’s  acquisition 
policies  and  process)  refers  to  a  program’s  best  cost  and  schedule  estimates  as  objective 
estimates. 

5MAIS  annual  reports  include  information  on  the  status  of  each  technical  performance 
target,  among  other  information.  Information  about  each  target  includes  a  description  of 
the  performance  characteristics,  the  objective  and  threshold  value  for  each  target  and, 
importantly,  whether  the  target  has  demonstrated  the  expected  performance. 
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Table  1:  MAIS  Programs  Assessed  Against  Leading  Practices 

Program  name 
Army 

Global  Combat  Supply  System  Increment  1  (GCSS-A  Inc  1) 

Integrated  Personnel  and  Pay  System  Increment  2  (IPPS-A  Inc  2) 

Air  Force 

Air  and  Space  Operations  Center-Weapon  System  Increment  10.2  (AOC-WS  Inc  10.2) 
Base  Information  Transport  Infrastructure  Wired  (BITI  Wired) 

Joint  Space  Operations  Center,  Mission  System  Increment  2  (JMS  Inc  2) 

Source:  DOD  data  |  GAO-17-322 


To  evaluate  DOD’s  approach  in  managing  the  acquisition  of  MAIS 
systems,  we  identified  risk  management  and  requirements  management 
practices  from  the  Software  Engineering  Institute’s  Capability  Maturity 
Model®  Integration  for  Acquisition  (CMMI-ACQ),  and  assessed  each  of 
the  programs  against  these  leading  practices.6  With  regard  to  risk 
management,  we  analyzed  each  program’s  key  documents,  such  as  risk 
management  plans,  risk  charts,  and  other  artifacts,  and  compared  them 
to  the  leading  practices.  For  requirements  management,  we  compared 
requirements  documents,  such  as  the  requirements  management  plan, 
traceability  matrix,  and  procedural  tools  to  the  leading  practices.7 

We  also  identified  key  systems  testing  and  integration  practices  from  the 
Institute  of  Electrical  and  Electronics  Engineers,  Incorporated  (IEEE),8 


6Software  Engineering  Institute,  Capability  Maturity  Model®  Integration  for  Acquisition 
(CMMI-ACQ),  Version  1.3  (Pittsburgh,  Pa.:  November  2010). 

7CMMI-ACQ;  and  GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for 
Developing  and  Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C.: 

March  2009);  Information  Technology:  DHS  Needs  to  Improve  Its  Independent  Acquisition 
Reviews,  GAO-11-581  (Washington,  D.C.:  July  28,  2011);  Information  Technology:  Critical 
Factors  Underlying  Successful  Major  Acquisitions,  GAO-12-7  (Washington,  D.C.:  Oct.  21, 
201 1 );  and  Schedule  Assessment  Guide:  Best  Practices  for  Project  Schedules, 

GAO-1 6-89G  (Washington,  D.C.:  December  201 5). 

8The  Institute  of  Electrical  and  Electronics  Engineers,  IEEE  International  Standard  for 
Software  and  Systems  Engineering,  IEEE  Standard  29119-1-2013™  (New  York,  NY: 

Sept.  1, 2013).  All  rights  reserved. 
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and  DOD’s  policy  and  guidance9  (DODI  5000.02,  Defense  Acquisition 
Guidebook,  and  Test  &  Evaluation  Management  Guide)  and  assessed 
each  of  the  five  programs  against  these  practices.  For  example,  we 
evaluated  the  test  manager  roles  and  responsibilities,  test  plans,  and  test 
reports  to  determine  the  extent  to  which  these  practices  were  applied.  We 
interviewed  program  officials  to  obtain  additional  information  on  each 
program’s  risk  and  requirements  management  processes,  and  systems 
testing  and  integration  practices. 

We  conducted  this  performance  audit  from  April  2016  to  March  2017  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  See  appendix  I  for  a  more 
detailed  discussion  of  our  objectives,  scope,  and  methodology. 


Background 


DOD’s  organizational  structure  includes  the  Office  of  the  Secretary  of 
Defense,  the  Joint  Chiefs  of  Staff,  the  military  services,  numerous 
defense  agencies  and  field  activities,  and  various  unified  combatant 
commands  that  contribute  to  the  oversight  of  DOD’s  MAIS  programs. 
Figure  1  provides  a  simplified  depiction  of  DOD’s  organizational  structure. 


department  of  Defense,  Operation  of  the  Defense  Acquisition  System,  DODI  5000.02 
(January  7,  2015);  Department  of  Defense,  Defense  Acquisition  Guidebook ;  Department 
of  Defense,  Test  and  Evaluation  Management  Guide  6th  Edition  (Washington,  D.C.: 
December  2012). 
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Figure  1:  Simplified  DOD  Organizational  Structure 


Source:  GAO  analysis  based  on  Department  of  Defense  data.  |  GAO-17-322 


The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD  (AT&L))  serves  as  the  Defense  Acquisition  Executive  and 
is  the  official  responsible  for  supervising  the  acquisition  and  oversight  of 
MAIS  programs.10  This  official  is  the  principal  acquisition  official  of  the 
department  and  is  the  acquisition  advisor  to  the  Secretary  of  Defense. 
The  USD  (AT&L)  has  policy  and  procedural  authority  for  the  defense 
acquisition  system,  which  establishes  the  steps  that  DOD  programs 
generally  take  as  the  department  plans,  designs,  acquires,  deploys, 
operates,  and  maintains  its  IT  systems  (discussed  in  more  detail  following 
this  section).  The  USD  (AT&L)’s  authority  also  includes  directing  the 
military  services  and  defense  agencies  on  acquisition  matters  and — 
unless  the  authority  is  delegated  to  the  service  or  agency — making 
milestone  decisions  for  MAIS  programs. 


10Effective  February  1 , 201 8,  the  position  of  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  will  be  eliminated  and  the  newly  created  positions  of  Under 
Secretary  of  Defense  for  Research  and  Engineering  and  Under  Secretary  of  Defense  for 
Acquisition  and  Sustainment  will  incorporate  duties  from  the  eliminated  position.  This 
change  is  per  the  National  Defense  Authorization  Act  for  Fiscal  Year  2017,  Pub.  L.  No. 
114-328,  §  901  (Dec.  23,  2016). 
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DOD’s  Acquisition 
Guidance  and  Framework 
for  Managing  MAIS 
Acquisitions 


The  Department  of  Defense  Instruction  5000. 0211  is  the  policy  that 
establishes  guidelines  for  the  management  of  all  acquisition  programs. 
The  department’s  policy  guidance  includes  six  defense  acquisition 
models  based  on  the  type  of  product  being  acquired  (e.g.,  software¬ 
intensive  programs  and  hardware-intensive  programs).  A  generic 
acquisition  model  that  shows  all  of  the  program  life-cycle  phases  and  key 
decision  points  is  shown  in  figure  2  and  described  below. 


Figure  2:  Generic  Model  from  the  Defense  Acquisition  System  Framework 
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Source:  GAO  analysis  based  on  Department  of  Defense  data.  |  GAO-17-322 


Materiel  solution  analysis:  Refine  the  initial  system  solution  (concept) 
and  create  a  strategy  for  acquiring  the  solution.  A  decision — referred  to  as 
Milestone  A,  the  risk  reduction  decision  point — is  made  at  the  end  of  this 
phase  to  authorize  entry  into  the  technology  maturation  and  risk  reduction 
phase. 


Technology  maturation  and  risk  reduction:  Determine  the  preferred 
technology  solution  and  validate  that  it  is  affordable,  satisfies  program 
requirements,  and  has  acceptable  technical  risk.  A  decision — referred  to 
as  Milestone  B,  the  program  initiation  decision  point — is  made  at  the  end 
of  this  phase  to  authorize  entry  of  the  program  into  the  engineering  and 
manufacturing  development  phase  and  award  development  contracts.  An 


^DOD  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System,  (Washington, 
D.C.:  Jan.  7,  2015). 
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acquisition  program  baseline12  is  first  established  at  the  Milestone  B 
decision  point.  A  program’s  first  acquisition  program  baseline  contains  the 
original  life-cycle  cost  estimate  (acquisition  and  operations  and 
maintenance  costs),  the  schedule  estimate  (major  milestones  and 
decision  points),  and  performance  parameters  that  were  approved  for  that 
program  by  the  Milestone  Decision  Authority.13  The  first  acquisition 
program  baseline  is  established  after  the  program  has  refined  user 
requirements  and  identified  the  most  appropriate  technology  solution  that 
demonstrates  that  it  can  meet  users’  needs. 

Engineering  and  manufacturing  development:  Develop  a  system  and 
demonstrate  through  testing  that  the  system  meets  all  program 
requirements.  A  decision — referred  to  as  Milestone  C,  the  deployment 
decision  point — is  made  during  this  phase  to  authorize  entry  of  the 
system  into  the  production  and  deployment  phase  or  into  limited 
deployment  in  support  of  operational  testing. 

Production  and  deployment:  Achieve  an  operational  capability  that 
meets  program  requirements,  as  verified  through  independent  operational 
tests  and  evaluation,  and  implement  the  system  at  all  applicable 
locations. 

Operations  and  support:  Operationally  sustain  the  system  in  the  most 
cost-effective  manner  over  its  life  cycle. 


12The  acquisition  program  baseline  is  developed  by  the  program  manager  and  approved 
by  the  Milestone  Decision  Authority  (the  official  responsible  for  deciding  whether  a 
program  meets  the  milestone  criteria  and  proceeds  to  the  next  phase  of  the  acquisition 
process)  before  the  formal  initiation  of  all  acquisition  programs  and  describes  the 
parameters  against  which  program  performance  will  be  measured  and  tracked.  The 
acquisition  program  baseline  states  the  threshold  and  objective  values  for  the  cost, 
schedule,  and  performance  requirements  fora  program.  An  acquisition  program  baseline 
is  required  for  each  increment  and  block  for  a  Major  Defense  Acquisition  Program.  Any 
deviations  from  an  approved  acquisition  program  baseline  must  be  documented. 

13Depending  on  the  program,  the  Milestone  Decision  Authority  can  be  the  Under 
Secretary  of  Defense  (Acquisition,  Technology,  &  Logistics),  the  head  of  the  relevant 
Department  of  Defense  component,  or  the  component  acquisition  executive. 
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Leading  Practices  for 
Managing  an  IT 
Acquisition  Program 


Entities  such  as  the  Project  Management  Institute,  the  Software 
Engineering  Institute  at  Carnegie  Mellon  University,  IEEE,  and  GAO  have 
developed  and  identified  leading  practices  to  help  guide  organizations  to 
effectively  plan  and  manage  the  acquisition  of  major  IT  systems,  such  as 
the  MAIS  programs.  Our  work  has  shown  that  proper  implementation  of 
such  practices  can  significantly  increase  the  likelihood  of  delivering 
promised  system  capabilities  on  time  and  within  budget.  These  practices 
include,  but  are  not  limited  to, 

•  Risk  management:  A  process  for  anticipating  problems  and  taking 
appropriate  steps  to  mitigate  risks  and  minimize  their  impact  on 
program  commitments.  It  involves  identifying  and  documenting  risks, 
categorizing  them  based  on  their  estimated  impact,  prioritizing  them, 
developing  risk  mitigation  strategies,  and  tracking  progress  in 
executing  the  strategies. 

•  Requirements  management:  Requirements  establish  what  the 
system  is  to  do,  how  well  it  is  to  do  it,  and  how  it  is  to  interact  with 
other  systems.  Appropriate  requirements  development  involves 
eliciting  and  developing  customer  and  stakeholder  requirements,  and 
analyzing  them  to  ensure  that  they  will  meet  users’  needs  and 
expectations.  It  also  consists  of  validating  requirements  as  the  system 
is  being  developed  to  ensure  that  the  final  systems  to  be  deployed  will 
perform  as  intended  in  an  operational  environment. 

•  Systems  testing  and  integration  management:  A  rigorous  and 
efficient  test  and  evaluation  program  provides  early  knowledge  of 
developmental  and  operational  issues.  Correcting  these  issues  early 
can  mitigate  risks  of  cost  overruns  and  schedule  slippages,  and  can 
ultimately  contribute  to  delivery  of  effective  and  suitable  systems  in  a 
timely  manner.  System  integration  ensures  that  all  the  “pieces”  of  the 
system  can  work  together  to  realize  the  program’s  goals.  It  involves 
the  collaborative  planning  and  execution  of  test  phases  and  events  to 
provide  shared  data  in  support  of  independent  analysis,  evaluation, 
and  reporting  by  all  stakeholders. 
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Most  Selected  MAIS 
Programs  Had 
Changes  in  Their 
Planned  Cost  or 
Schedule  Estimates, 
and  Half  Met  Their 
Technical 

Performance  Targets 


The  majority  of  the  18  selected  MAIS  programs  that  we  reviewed  had 
experienced  changes  in  their  planned  cost  estimates  and  in  their  planned 
schedule  estimates  when  comparing  the  first  acquisition  program 
baseline  to  the  most  recent  acquisition  program  baseline  estimate.14 
These  cost  estimate  changes  ranged  from  a  decrease  of  $1,466.2  million 
(-39  percent)  to  an  increase  of  $1,625.7  million  (469  percent),  and 
schedule  estimate  changes  ranged  from  a  slippage  of  13  years  and  9 
months  to  no  change.  Further,  nine  of  the  programs  had  met  all  of  their 
technical  performance  targets,  while  four  of  the  programs  had  partially 
met  the  performance  targets.  The  remaining  five  programs  had  not  yet 
conducted  testing  activities.  As  a  result,  it  is  too  early  to  report  on  the 
status  of  technical  performance  targets.  Table  2  shows  the  extent  of 
changes  in  planned  cost  and  schedule  estimates  for  each  of  the  selected 
MAIS  programs  we  reviewed  since  the  first  program  baseline  estimate 
and  the  number  of  technical  performance  targets  met.  See  appendix  II  for 
the  detailed  profiles  of  each  program. 


Table  2:  Summary  of  Changes  in  Cost  and  Schedule  Estimates  and  the  Status  of  Technical  Performance  Targets  for  the  18 
Selected  Major  Automated  Information  System  Programs  (from  the  first  acquisition  baseline  estimate) 

Program  name 

Change  in 
cost  estimate 
(dollars  in 
millions) 

Change  in 
schedule  estimate 
(delay) 

Number  of 
performance 
targets  met 

Army 

•  Tactical  Mission  Command  (TMC) 

$1,421.7  (159%) 

3  months 

3  of  3 

•  Integrated  Personnel  and  Pay  System-Army  Increment  1 
(IPPS-AIncI) 

$10.5  (3%) 

6  months 

3  of  3 

•  Integrated  Personnel  and  Pay  System-Army  Increment  2 
(IPPS-A  Inc  2) 

$330.9  (19%) 

No  change 

Tests  not  yet 
conducted 

•  Logistics  Modernization  Program  Increment  2  (LMP  Inc  2) 

$6.9  (1  percent) 

5  months 

7  of  7 

•  Global  Combat  Support  System-Army  (GCSS-A  Inc  1 ) 

-$58.5  (-1.5%) 

10  months 

2  of  2 

Air  Force 

•  Defense  Enterprise  Accounting  and  Management  System 
Increment  1  (DEAMS  Inc  1) 

$124.9  (9%) 

1  year 

6  of  9 

14 

The  acquisition  program  baseline  is  developed  by  the  program  manager  before  the 
initiation  of  a  program  for  all  acquisition  programs  and  depicts  the  current  condition  of  a 
program.  The  plan  (which  must  be  approved  by  the  Milestone  Decision  Authority)  states 
the  threshold  and  objective  values  for  the  cost,  schedule,  and  performance  requirements 
fora  program.  An  acquisition  program  baseline  is  required  for  each  increment  and  block 
for  a  MAIS  or  Major  Defense  Acquisition  Program.  Any  deviations  from  an  approved 
acquisition  program  baseline  must  be  documented. 
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Program  name 

Change  in 
cost  estimate 
(dollars  in 
millions) 

Change  in 
schedule  estimate 
(delay) 

Number  of 
performance 
targets  met 

•  Integrated  Strategic  Planning  and  Analysis  Network 

Increment  4  (ISPAN  Inc  4) 

No  change 

No  change 

Tests  not  yet 
conducted 

•  Air  and  Space  Operations  Center-Weapon  System  Inc  10.2 
(AOC-WS  Inc  10.2) 

$56.8  (1%) 

3  years 

Tests  not  yet 
conducted 

•  Joint  Space  Operations  Center  Mission  System  Increment  2 
(JMS  Inc  2) 

-$134.6  (-12%) 

2  years  10  months 

Tests  not  yet 
conducted 

•  Base  Information  Transport  Infrastructure  Wired  (BITI  Wired) 

-$1,466.2  (-39%) 

2  months 

Oof  4a 

Navy 

•  Common  Aviation  Command  and  Control  System  Increment 

1  (CAC2S  Inc  1) 

$1,625.7  (469%) 

13  years  9  months 

2  of  2 

•  Global  Combat  Support  System-Marine  Corps  Logistics 

Chain  Management  Increment  1  (GCSS-MC  LCM  Inc  1) 

$1,360.3  (295%) 

5  years  7  months 

2  of  2 

•  Consolidated  Afloat  Network  Enterprise  Services  (CANES) 

-$830.3  (-7%) 

2  years 

8  of  9 

Defense  Information  Systems  Agency 

•  Teleport  Generation  3  (Teleport  Gen  3) 

$11.6(2%) 

3  years  2  months 

1  of  3 

Defense  Health  Agency 

•  Department  of  Defense  Healthcare  Management  System 
(DHMSM) 

No  change 

No  change 

Tests  not  yet 
conducted 

•  integrated  Electronic  Health  Record  Increment  1 
(iEHR  Inc  1) 

-$718.3  (-70%) 

11  months 

3  of  3 

Defense  Logistics  Agency 

•  Defense  Agencies  Initiative  Increment  2  (DAI  Inc  2) 

$2.6(1%) 

No  change 

5  of  5 

Department  of  Defense 

•  Theater  Medical  Information  Program-Joint  Increment  2 
(TMIP-J  Inc  2) 

$374.9  (161%) 

5  years  6  months 

10  of  10 

Source:  GAO  analysis  of  data  provided  by  DOD  officials  |  GAO-17-322 

aWe  categorized  BITI  Wired  as  partial  because  it  has  met  4  of  4  targets  at  167  of  178  total  sites. 

Of  the  18  MAIS  programs  we  reviewed,  16  had  experienced  changes  in 
their  planned  cost  estimates.  Specifically,  five  of  the  16  programs  had 
cost  decreases  that  ranged  from  $58.5  million  (-1.5  percent)  for  the 
Army’s  Global  Combat  Support  System,  Increment  1  (GCSS-A  Inc  1) 
program  to  $1,466.2  million  (-39  percent)  for  the  Air  Force’s  Base 
Information  Transport  Infrastructure  Wired  (BITI  Wired)  program. 

Officials  reported  various  reasons  for  the  decreases  in  planned  cost 
estimates.  For  example,  the  Defense  Health  Agency’s  integrated 
Electronic  Health  Record  Increment  1  (iEHR  Inc  1)  program  reported  that 
the  70  percent  cost  decrease  was  due  to  a  reduction  in  the  program’s 


Sixteen  of  Eighteen 
Selected  MAIS  Programs 
Had  Changes  in  Their 
Planned  Cost  Estimates 
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scope.  Officials  for  the  Air  Force’s  BITI  Wired  program  reported  that  the 
39  percent  cost  decrease  was  due  to  early  assumptions  about  risks  not 
materializing.  For  example,  when  the  program  had  solicited  bids  for  the 
contract,  they  included  risk  factors  that  never  materialized. 

In  addition,  11  of  the  16  programs  had  experienced  cost  increases.  These 
increases  ranged  from  $2.6  million  (1  percent)  for  the  Defense  Logistics 
Agency’s  Defense  Agencies  Initiative  Increment  2  (DAI  Inc  2)  program  to 
$1,625.7  million  (469  percent)  for  the  Navy’s  Common  Aviation  Command 
and  Control  System  Increment  1  (CAC2S  Inc  1 )  program.  Among  the  1 1 
programs  that  reported  an  increase,  the  average  increase  was  $457.2 
million. 

Program  officials  reported  a  variety  of  reasons  for  the  increases  in 
planned  cost  estimates.  For  example,  officials  for  the  Army’s  Tactical 
Mission  Command  (TMC)  program  attributed  its  159  percent  increase  to, 
in  part,  higher  than  expected  costs  for  research  and  development  testing. 
In  addition,  officials  for  the  Air  Force’s  Defense  Enterprise  Accounting  and 
Management  System  Increment  1  (DEAMS  Inc  1)  program  attributed  a  9 
percent  cost  increase  to  the  program’s  growth  in  scope  and  the  addition 
of  software  upgrade  enhancements.  Further,  officials  for  the  Navy’s 
CAC2S  system  attributed  cost  increases  of  469  percent  to  the  program’s 
early  developmental  challenges,  which  included  program  scope  growth 
and  restructuring. 

Two  programs  had  not  experienced  any  changes  in  their  cost  estimates: 
These  include  the  Defense  Health  Agency’s  Department  of  Defense 
Healthcare  Management  System  (DHMSM)  program  and  the  Air  Force’s 
Integrated  Strategic  Planning  and  Analysis  Network  Increment  4  (ISPAN 
Inc  4)  program. 


Fourteen  of  Eighteen 
Selected  MAIS  Programs 
Had  Changes  in  Their 
Planned  Schedule 
Estimate 


Of  the  18  selected  MAIS  programs  we  reviewed,  14  programs  had 
experienced  changes  in  their  planned  schedule  estimates.  These 
changes  consisted  of  schedule  slippages  that  ranged  from  2  months  for 
the  Air  Force’s  BITI  Wired  program  to  13  years  and  9  months  for  the 
Navy’s  CAC2S  program. 

Program  officials  reported  various  reasons  for  the  changes  in  their 
planned  schedule  estimates.  For  example,  officials  for  the  Navy’s  CAC2S 
program  attributed  the  schedule  slippages  to  the  addition  of  new 
requirements  and  program  restructuring.  Officials  for  the  Air  Force’s  AOC- 
WS  Inc  10.2  program  reported  that  the  3  years  of  schedule  slippages 
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were  due  to  a  combination  of  test  problems  and  system  maturity  issues. 
In  addition,  officials  for  the  Air  Force’s  JMS  Inc  2  system  cited  a  larger 
than  expected  number  of  discrepancies  in  testing  that  had  caused 
schedule  delays  totaling  2  years  and  10  months.  In  turn,  the  program 
delayed  completion  of  its  Milestone  C  and  Full  Deployment  Decision  by 
more  than  1  year  beyond  the  original  estimate. 

Four  MAIS  programs  experienced  no  change  to  their  planned  schedule 
estimates — the  DAI  Inc  2,  DHMSM,  IPPS-A  Inc  2,  and  the  ISPAN  Inc  4 
programs. 


Nine  Selected  MAIS 
Programs  Had  Met 
Technical  Performance 
Targets,  while  Others 
Were  at  Various  Stages 
of  Progress 


Among  other  information,  DOD  uses  key  performance  parameters  as  a 
metric  to  report  on  each  program’s  progress  toward  meeting  the  system 
technical  performance  targets.  This  information  includes  a  description  of 
the  performance  characteristics,  the  objective  and  threshold  value  for 
each  target  and,  importantly,  whether  the  target  has  been  met  in 
demonstrating  performance.  For  example,  one  program  identified 
“network  readiness  and  exchange  of  information”  as  a  technical 
performance  target.  The  program  also  established  an  objective  value  for 
this  target  as  successfully  processing  98  percent  of  the  information  into 
the  system,  and  established  an  threshold  value  for  this  target  as 
successfully  processing  95  percent  of  the  information  into  the  system. 

Among  the  18  selected  MAIS  programs  we  reviewed  that  had  established 
technical  performance  targets,  nine  had  met  all  of  their  defined  targets. 

As  an  example  of  technical  performance  targets  that  had  been  met,  the 
Army’s  GCSS  Inc  1  program  reported  in  the  2016  Major  Automated 
Information  System  Annual  Report  that  both  of  its  technical  performance 
targets  had  been  met.  These  targets  related  to  the  readiness  of  the 
system  to  fully  support  operational  activities  and  information  exchanges  in 
the  architecture  (referred  to  as  Net  Ready)  and  compliance  with  the 
operational  activities  that  represent  the  system’s  mission  critical  functions 
(referred  to  as  Mission  Critical  Functions). 

As  another  example,  the  Defense  Health  Agency’s  iEHR  Inc  1  program 
reported  that  all  3  performance  targets  had  been  met.  These  targets 
related  to  ensuring  that  end  user  devices  display  the  same  information, 
context,  and  display  of  the  patient,  expediting  user  access  to  the 
applications,  and  ability  of  users  to  move  from  one  device  to  another. 

Further,  of  the  18  MAIS  programs,  four  programs — the  Air  Force’s  BITI- 
Wired  and  DEAMS  Inc  1  programs,  Navy’s  CANES  and  Defense 
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Information  Systems  Agency’s  Teleport  Gen  3  programs — had  partially 
met  their  technical  performance  targets.  For  example,  BITI-Wired  had 
partially  implemented  all  4  of  its  performance  targets — related  to 
readiness  of  the  network,  availability  of  redundant  features  such  as  dual 
power  supplies  and  spares,  reliability  of  systems,  and  the  installation  of 
support  equipment  at  all  178  sites.  According  to  program  officials,  support 
equipment  had  been  installed  at  167  of  the  178  total  sites  and  the 
program  expects  all  4  performance  targets  to  be  met  in  September  2017, 
the  planned  full  deployment  date.  As  another  example,  the  Air  Force’s 
DEAMS  Inc  1  program  officials  reported  that  6  of  9  targets  had  been  met. 
For  the  3  performance  targets  that  had  not  been  met,  officials  reported 
that  processing  issues  had  prevented  them  from  meeting  these  targets, 
which  included  establishing  accurate  fund  balances,  reporting  in  a  timely 
manner,  and  ensuring  readiness  of  the  network.  As  of  June  2016, 
program  officials  reported  they  were  working  to  address  these  issues. 

Finally,  five  programs — IPPS-A  Inc  2,  AOC-WS  Inc  10.2,  ISPAN  Inc  4, 
JMS  Inc  2,  and  DHMSM — were  in  the  early  stages  of  the  system 
development  and  had  not  begun  testing.  As  a  result,  it  is  too  early  to 
determine  the  status  of  performance  targets  because  information  on  the 
status  of  technical  performance  is  not  yet  available. 
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All  Five  Selected 
MAIS  Programs  Had 
Fully  or  Partially 
Implemented  Leading 
Practices  for 
Managing  Risk, 
Requirements,  and 
Systems  Testing  and 
Integration 


According  to  the  Software  Engineering  Institute’s  Capability  Maturity 
Model®  Integration  for  Acquisition  (CMMI-  ACQ),15  leading  practices  for 
managing  an  IT  program  include  risk  management,  and  requirements 
management.  Effective  requirements  management  includes  establishing 
an  agreed-upon  set  of  requirements  and  ensuring  traceability  between 
requirements  and  work  products.  Further,  changes  to  requirements 
should  be  managed  in  collaboration  with  stakeholders  for  the  acquisition 
of  systems  within  their  established  cost,  schedule,  and  delivery  of  the 
expected  product.  Likewise,  an  effective  risk  management  process 
includes  identifying  problems  before  they  occur,  so  that  risk-handling 
activities  may  be  planned  and  invoked,  as  needed,  across  the  life  of  the 
project  in  order  to  mitigate  the  potential  for  adverse  impacts.  Further, 
regarding  systems  testing  and  integration,  other  leading  practices  from 
the  Institute  of  Electrical  and  Electronics  Engineers,  Incorporated  (IEEE) 
and  DOD  policy  guidance16  (DOD  Instruction  5000.02,  Defense 
Acquisition  Guidebook,  and  Test  &  Evaluation  Management  Guide)  state 
that  roles  and  responsibilities  should  be  established  and  test-related 
plans,  schedules,  and  reports  should  be  developed  to  better  manage  test 
and  integration  activities. 


All  five  MAIS  programs  that  we  selected  for  an  in-depth  review  had  fully 
or  partially  implemented  leading  practices  for  managing  risk, 
requirements,  and  systems  testing  and  integration.  Table  3  shows  the 
extent  to  which  these  programs  had  implemented  the  practices  for 
requirements  management,  risk  management,  and  systems  testing  and 
integration  management.  We  assessed  one  program  as  “not  applicable” 
because  it  was  initiated  after  a  limited  production  decision  point  (post- 
Milestone  C)  and,  therefore,  had  not  implemented  certain  practices. 


15Software  Engineering  Institute,  Capability  Maturity  Model®  Integration  for  Acquisition 
(CMMI-ACQ),  Version  1.3  (Pittsburgh,  Pa.:  November  2010;  November  2010). 

16Department  of  Defense,  Operation  of  the  Defense  Acquisition  System,  DODI  5000.02 
(January  7,  2015);  Department  of  Defense,  Defense  Acquisition  Guidebook ;  Department 
of  Defense,  Test  and  Evaluation  Management  Guide  6th  Edition  (December  2012). 
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Table  3:  Extent  to  Which  Selected  Major  Automated  Information  Systems  Programs  Applied  Leading  Practices  for 
Requirements  and  Risk  Management  and  Systems  Testing  and  Integration 


Leading  Practice 

Air  Force  AOC- 
WS  Inc  10.2 

Air  Force  BITI 
Wired 

Air  Force  JMS 
Inc  2 

Army  GCSS-A 
Inc  1 

Army  IPPS-A 
Inc  2 

Requirements  management 

1 .  Develop  an  understanding  with  the 
requirements  providers  on  the 
meaning  of  the  requirements 

• 

• 

• 

3 

• 

2.  Obtain  commitment  to 
requirements  from  project 
participants 

• 

N/A 

• 

3 

• 

3.  Manage  changes  to  requirements 
as  they  evolve  during  the  project 

3 

• 

• 

3 

• 

4.  Maintain  bidirectional  traceability 
among  requirements  and  work 
products 

• 

N/A 

• 

3 

• 

5.  Ensure  that  project  plans  and  work 
products  remain  aligned  with 
requirements 

• 

N/A 

• 

3 

• 

Risk  management 

1 .  Determine  risk  sources  and 
categories  used  to  examine  and 
oversee  changes  that  impact  the 
project. 

• 

• 

• 

• 

• 

2.  Define  parameters  used  to  analyze 
and  categorize  risks  and  to  control 
the  risk  management  effort. 

• 

• 

• 

3 

• 

3.  Establish  and  maintain  the 
strategy  to  be  used  for  risk 
management. 

3 

• 

• 

• 

• 

4.  Identify  and  document  risks  that 
could  negatively  affect  work 
efforts. 

3 

• 

• 

• 

• 

5.  Evaluate  and  categorize  each 
identified  risk  using  defined  risk 
categories  and  parameters,  and 
determine  its  relative  priority. 

• 

• 

• 

• 

• 

6.  Develop  a  risk  mitigation  plan  in 
accordance  with  the  risk 
management  strategy. 

3 

• 

• 

• 

• 

7.  Monitor  the  status  of  each  risk 
periodically  and  implement  the  risk 
mitigation  plan  as  appropriate. 

• 

• 

• 

3 

• 

Systems  Testing  &  Integration 
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Leading  Practice 

Air  Force  AOC- 
WS  Inc  10.2 

Air  Force  BITI 
Wired 

Air  Force  JMS 
Inc  2 

Army  GCSS-A 
Inc  1 

Army  IPPS-A 
Inc  2 

1.  Establish  roles  and  responsibilities 
to  manage  testing/integration 
activities. 

• 

• 

O 

• 

• 

2.  Formulate  test  related  plans, 
schedules,  and  reports. 

• 

• 

• 

• 

• 

Source:  GAO  analysis  of  data  provided  by  DOD  officials  |  GAO-17-322 


Legend:  AOC-WS  Inc  10.2  =  Air  and  Space  Operations  Center-Weapon  System  Increment  10.2;  BITI 
Wired  =  Base  Information  Transport  Infrastructure  Wired;  JMS  Inc  2  =  Joint  Space  Operations 
Center,  Mission  System  Increment  2;  GCSS-A  Inc  1  =  Global  Combat  Support  System-Army 
Increment  1;  IPPS-A  Inc  2  =  Integrated  Personnel  and  Pay  System — Army  Increment  2. 

•  -Fully  implemented  indicates  that  the  agency  provided  evidence  that  it  had  addressed  all  of  the  sub 
practices  in  a  category 

o  -  Partially  implemented  indicates  that  the  agency  provided  evidence  that  it  had  addressed  at  least 
one,  but  not  all,  of  the  sub  practices  in  a  category 

N/A  -  Not  applicable  indicates  that  the  agency  did  not  complete  this  practice  because  the  program 
was  initiated  post-Milestone  C. 


Two  of  the  Five  MAIS 
Programs  Had  Not  Fully 
Implemented  All  Practices 
for  Managing 
Requirements 


Leading  requirements  management  practices  include  establishing  an 
agreed-upon  set  of  requirements,  ensuring  traceability  between 
requirements  and  work  products,  and  managing  any  changes  to  the 
requirements  in  collaboration  with  stakeholders.  Such  leading  practices 
help  organizations  to  better  manage  the  design,  development,  and 
delivery  of  systems  within  established  cost  and  schedule  time  frames. 
These  practices  include 


•  developing  an  understanding  with  the  requirements  providers  on  the 
meaning  of  the  requirements, 

•  obtaining  commitment  to  requirements  from  project  participants, 

•  managing  changes  to  requirements  as  they  evolve  during  the  project, 

•  maintaining  bidirectional  traceability  among  requirements  and  work, 
and 

•  ensuring  that  project  plans  and  work  products  remain  aligned  with 
requirements. 

Three  of  the  five  programs  had  fully  implemented  the  requirements 
management  practices,  while  the  other  two  had  partially  implemented 
some  practices. 


Page  16 


GAO-17-322  DOD  Major  Automated  Information  Systems 


Air  Force —  Air  and  Space  Operations  Center-Weapon  System 
Increment  10.2  (AOC-WS  Inc  10.2) 

The  Air  Force  had  fully  implemented  four,  and  partially  implemented  one, 
of  the  five  requirements  management  practices  for  the  AOC  WS  Inc  10.2 
program.  For  example,  the  program  developed  an  understanding  with 
requirements  providers  on  the  meaning  of  the  requirements  and 
established  a  requirements  working  group  that  oversees  the  development 
and  management  of  program  requirements.  In  addition,  the  AOC-WS  Inc 
10.2  program  obtained  commitment  to  requirements  from  project 
participants  through  the  DOD  configuration  control  board  process,  in 
which  requirements  were  reviewed  and,  upon  acceptance,  approved, 
documented,  and  tracked.  The  program  also  managed  changes  to 
requirements  as  they  evolved  during  the  project;  however,  although 
changes  to  requirements  were  documented  and  maintained,  the  program 
did  not  provide  evidence  of  its  rationale  for  the  changes.  The  AOC-WS 
Inc  10.2  program  has  since  completed  its  acquisition  phase  and, 
therefore,  providing  a  rationale  for  changes  made  during  the  requirements 
development  phase  would  not  impact  the  program  moving  forward. 

Air  Force —  Base  Information  Transport  Infrastructure  Wired  (BITI 
Wired) 

The  Air  Force  had  fully  implemented  the  two  applicable  requirements 
management  practices  for  the  BITI  Wired  program.  Specifically,  the 
program  had  developed  an  understanding  with  requirements  providers  on 
the  meaning  of  the  requirements  and  had  managed  changes  to 
requirements  as  they  evolved  during  the  project.  The  BITI  Wired 
requirements  management  plan  established  the  criteria  for  the  evaluation 
and  acceptance  of  the  requirements. 

On  the  other  hand,  three  of  the  five  requirements — obtain  commitment  to 
requirements  from  project  participants;  maintain  bidirectional  traceability 
among  requirements  and  work  products;  and  ensure  that  project  plans 
and  work  products  remain  aligned  with  requirements — were  not 
applicable  for  the  BITI  Wired  Program.  This  was  due  to  the  Air  Force 
beginning  the  program  after  Milestone  C. 

Air  Force — Joint  Space  Operations  Center,  Mission  System 
Increment  2  (JMS  Inc  2) 

The  Air  Force  had  fully  implemented  the  five  requirements  management 
practices  for  the  JMS  Inc  2  program.  Specifically,  the  program  had 
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developed  an  understanding  with  requirements  providers  on  the  meaning 
of  the  requirements.  For  example,  the  program’s  Requirements 
Engineering  and  Management  Plan  established  criteria  for  distinguishing 
appropriate  requirements  and  for  evaluating  and  accepting  requirements. 
In  addition,  changes  to  requirements  were  managed  as  they  evolved 
during  the  project.  For  example,  requirements  change  requests  had  been 
sent  through  the  Lead  Command,  which  validated  them  and  sent  them  to 
the  Program  Management  Office  for  approval.  Lastly,  the  program 
maintained  requirements  traceability  from  source  requirements  to  lower- 
level  requirements  to  ensure  that  all  source  requirements  had  been 
completely  addressed.  Specifically,  the  program  had  established 
processes  for  managing  the  program’s  Requirements  Traceability  Matrix 
database,  which  included  activities  to  enter  approved  requirements, 
sequence  traceability  changes,  update  development,  test  status,  and 
report  data. 

Army —  Global  Combat  Support  System-Army  Increment  1  (GCSS-A 
Inc  1) 

The  Army  had  partially  implemented  all  five  requirements  management 
practices  for  the  GCSS-A  Inc  1  program.  Officials  reported  that,  although 
the  program  had  progressed  to  the  sustainment  phase,  and  its 
requirements  activities  had  been  completed  since  2014,  the  Configuration 
Steering  Board  review  process  had  been  used  to  manage,  oversee,  and 
approve  requirements  as  they  evolved  during  the  project.  The  program 
had  developed  a  Software  Development  Overview  Schedule  that 
provided  a  general  overview  of  their  software  development  life  cycle 
process. 

Nevertheless,  an  approved  and  signed  Requirements  Management  Plan 
at  program  initiation  had  not  been  implemented.  Such  a  plan  would  have 
been  important  to  ensuring  an  understanding  with  the  requirements 
providers  of  the  meaning  of  the  requirements.  Further,  the  program’s 
Release  Management  Plan  included  a  section  on  the  requirements 
process  and  references  to  a  central  information  repository  in  the 
Requirements  Traceability  database.  However,  the  requirements 
traceability  matrix  provided  did  not  contain  all  relevant  information  such 
as  “priority.”  These  important  management  tools  could  have  provided  the 
program  with  the  discipline,  structure,  and  oversight  typically  associated 
with  successful  organizations  that  apply  this  practice.  However,  since  the 
GCSS-A  Inc  1  program  has  completed  its  acquisition  phase,  fully 
implementing  these  practices  now  would  not  impact  the  program  moving 
forward. 
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Army — Integrated  Personnel  and  Pay  System — Army  Increment  2 
(IPPS-A  Inc  2) 

The  Army  had  fully  implemented  the  five  requirements  management 
practices  for  the  IPPS-A  Inc  2  program.  Specifically,  the  program  had 
developed  an  understanding  with  requirements  providers  on  the  meaning 
of  the  requirements  and  had  managed  changes  to  requirements  as  they 
evolved  during  the  project.  For  example,  the  program’s  Requirements 
Management  Plan  described  the  criteria  and  process  through  which 
requirements  are  evaluated  and  accepted.  In  addition,  the  IPPS-A 
Requirements/Configuration  Control  Board  Charter  requires  that  all 
changes  be  vetted  through  senior  governance  boards,  such  as  the 
Council  of  Colonels  and  General  Officer’s  Steering  Committee.  Further, 
the  program  maintained  bidirectional  traceability  of  requirements. 
Specifically,  a  history  for  all  requirements  changes  was  maintained  within 
a  repository  for  requirements  that  were  managed  by  the  Application 
Lifecycle  Management  Tool. 


Two  of  the  Five  MAIS 
Programs  Had  Not  Fully 
Implemented  All  Practices 
for  Managing  Risk 


According  to  leading  practices,  an  effective  risk  management  process 
identifies  potential  problems  before  they  occur,  so  that  risk-handling 
activities  may  be  planned  and  invoked,  as  needed,  across  the  life  of  the 
project  in  order  to  mitigate  the  potential  for  adverse  impacts.  These 
practices  include 


•  determining  risk  sources  and  categories; 

•  defining  parameters  used  to  analyze  and  categorize  risks  and  to 
control  the  risk  management  effort; 

•  establishing  and  maintaining  the  strategy  to  be  used  for  risk 
management; 

•  identifying  and  documenting  risks; 

•  evaluating  and  categorizing  each  identified  risk  using  defined  risk 
categories  and  parameters,  and  determining  its  relative  priority; 

•  developing  a  risk  mitigation  plan  in  accordance  with  the  risk 
management  strategy;  and 

•  monitoring  the  status  of  each  risk  periodically  and  implementing  the 
risk  mitigation  plan  as  appropriate. 

Three  of  the  five  programs  had  fully  implemented  the  risk  management 

practices,  while  two  had  partially  implemented  some  practices. 
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Air  Force— AOC-WS  Inc  10.2 


The  Air  Force  had  fully  implemented  four,  and  partially  implemented 
three,  of  the  seven  risk  management  practices  for  the  AOC  WS  Inc  10.2 
program.  For  example,  the  program  had  determined  risk  sources  and 
categories  used  to  examine  and  oversee  changes  that  impact  the  project, 
as  described  in  the  program’s  Risk  Management  Plan.  The  program  also 
had  defined  parameters  used  to  analyze  and  categorize  risks  and  to 
control  the  risk  management  effort.  Specifically,  the  Risk  Management 
Plan  evaluated  and  assigned  values  to  each  risk  by  calculating  the 
probability  and  consequence  of  a  risk,  as  well  as  its  likelihood. 

The  AOC-WS  Inc  10.2  program  had  partially  implemented  the  key 
management  practice  to  identify  risks  that  could  negatively  affect  work 
efforts.  For  example,  although  the  program  identified  risk  in  specific  risk 
categories,  it  did  not  identify  risks  that  were  outside  the  normal  scope  of 
the  project. 

In  addition,  while  the  program  had  developed  criteria  for  evaluating  and 
qualifying  risk  likelihood  and  severity  levels,  it  did  not  develop  an  overall 
risk  mitigation  plan  in  accordance  with  the  risk  management  strategy. 

This  occurred  because  the  program  underwent  a  critical  change  and, 
according  to  program  officials,  this  critical  change  altered  the 
management  of  the  program  due  to  its  complexity.  Until  an  overall  risk 
mitigation  plan  is  developed,  the  program  will  not  be  able  to  effectively 
manage  risk  mitigation  and  contingency  planning  activities. 

Air  Force — BITI  Wired 

The  Air  Force  had  fully  implemented  all  seven  of  the  risk  management 
practices  for  the  BITI  Wired  program.  Specifically,  the  program  had 
defined  parameters  used  to  analyze  and  categorize  risks  and  to  control 
the  risk  management  effort.  For  example,  the  program  evaluated  and 
assigned  values  to  each  risk  by  calculating  the  probability  and 
consequence  of  a  risk  as  well  as  its  likelihood  and  consequence.  The 
program  had  also  identified  and  documented  risks  that  could  negatively 
affect  work  efforts.  For  example,  the  Risk  Management  Plan  established 
risk  conditions,  thresholds,  and  probabilities  of  occurrence.  In  addition, 
the  BITI  Wired  program  monitored  the  status  of  each  risk  periodically  and 
implemented  risk  mitigation  plans  as  appropriate  through  the  use  of  a  risk 
management  tool  and  risk  board. 
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Air  Force — JMS  Inc  2 


The  Air  Force  had  fully  implemented  all  seven  of  the  risk  management 
practices  for  the  JMS  Inc  2  program.  Specifically,  the  program  had 
defined  parameters  used  to  analyze  and  categorize  risks  and  to  control 
the  risk  management  effort.  For  example,  as  detailed  in  the  program’s 
Risk  and  Opportunity  Management  Plan,  risk  ratings,  including  likelihood 
and  consequence  of  occurrence,  were  established  through  the  risk 
matrix;  this  provided  a  consistent,  standardized  method  for  risk  analysis. 
In  addition,  defined  thresholds  were  used  to  prioritize  risks  and  trigger 
management  action.  Further,  the  program  had  developed  risk 
management  plans  establishing  a  strategy  used  for  risk  management. 
Specifically,  risks  were  monitored  and  risk  status  was  reviewed  in  the 
issue  database  as  well  as  through  meetings  used  to  report  on 
programmatic  metrics.  Standard  operating  procedures  had  also  been 
developed  to  manage  risk,  including  one  to  manage  risk  identification, 
analysis,  mitigation,  and  tracking. 

Army — GCSS-A  Inc  1 

The  Army  had  fully  implemented  five,  and  partially  implemented  two,  of 
the  risk  management  practices  for  the  GCSS-A  Inc  1  program. 
Specifically,  the  program  had  a  Risk  and  Opportunity  Management  Plan 
that  identified  risk  sources  and  categories.  Further,  in  order  to  control  the 
risk  management  effort,  risks  were  categorized  and  grouped  according  to 
defined  risk  categories  in  the  program’s  Risk  Management  Plan.  Further, 
identified  risks  were  evaluated  using  defined  risk  parameters.  For 
example,  the  Risk  Register  identified  risk  consequence  and  probability. 

On  the  other  hand,  the  program  did  not  define  thresholds  or  bounds  for 
mitigation  or  management  action.  According  to  leading  practices,  the  risk 
management  effort  can  be  prioritized,  controlled,  and  managed  better  by 
defining  thresholds  and  categorizing  risks  for  mitigation/management 
action.  Although  program  officials  stated  that  they  were  in  the  process  of 
continuing  to  optimize  their  risk  management  efforts,  and  were  updating 
their  standard  operating  procedures,  this  key  leading  practice  has  not  yet 
been  implemented.  Until  the  program  has,  and  is  guided  by,  updated 
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procedures,  it  will  not  be  positioned  to  ensure  that  risks  can  be  effectively 
prioritized,  controlled,  and  managed.17 

Army— IPPS-A  Inc  2 

The  Army  had  fully  implemented  all  seven  of  the  risk  management 
practices  for  the  IPPS-A  Inc  2  program.  Specifically,  the  program’s  Risk 
Management  Plan  provided  guidelines  that  determined  risk  sources  and 
categories.  These  risks  were  categorized  in  the  Risk  Management  Tool 
based  on  the  “Risk  Taxonomy”  found  in  the  Risk  Management  Plan. 
Specifically,  the  risks  were  assigned  to  three  main  categories — technical, 
programmatic,  and  external  risks.  The  program  also  ensured  potential 
risks  were  identified  and  documented.  For  example,  the  Defense 
Information  Systems  Agency,  which  hosts  the  two  IPPS-A  Defense 
Enterprise  Computing  Centers  (in  St.  Louis,  MO,  and  Ogden,  UT), 
conducted  assessments  to  ensure  the  sites  were  in  compliance  with 
environmental  controls.  Further,  the  program  developed  risk  mitigation 
plans  that  identified  the  person  or  group  responsible  for  addressing  each 
risk  through  the  Risk  Register. 


Four  of  the  Five  MAIS 
Programs  Had  Fully 
Implemented  All  Practices 
for  Managing  Systems 
Testing  and  Integration 

•  establishing  roles  and  responsibilities  to  manage  testing  and 
integration  activities  and 

•  developing  test-related  plans,  schedules,  and  reporting. 

Four  of  the  five  programs  had  fully  implemented  practices  for  systems 
testing  and  integration. 


According  to  leading  practices  for  managing  systems  testing  and 
integration  activities,  roles  and  responsibilities  should  be  established  and 
test-related  plans,  schedules,  and  reports  should  be  developed.  Doing  so 
provides  a  baseline  by  which  to  measure  progress  so  that  appropriate 
corrective  actions  can  be  taken  when  the  program’s  performance 
deviates  from  the  plans.  These  practices  include 


17Subsequent  to  sending  a  draft  of  this  report  to  DOD  for  comment  in  February  2015,  the 
Army’s  GCSS-A  Inc  1  program  manager  amended  their  Risk  Management  Plan  for  Project 
Manager  Army  Enterprise  Systems  Integration  Program  on  March  1 , 2017.  This 
acquisition  decision  brought  the  program  back  into  compliance  with  this  leading  practice. 
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Air  Force— AOC-WS  Inc  10.2 


The  Air  Force  had  fully  implemented  both  systems  testing  and  integration 
practices  for  the  AOC-WS  Inc  10.2  program.  The  program  established 
roles  and  responsibilities  to  manage  testing/integration  activities  and 
formulated  test  related  plans,  schedules,  and  reports.  Specifically,  the 
program’s  Integrated  Test  Team  Charter  and  the  Test  and  Evaluation 
Master  Plan  (TEMP)  established  organizational  and  individual  roles  and 
responsibilities  fortesting  and  integration.  Further,  the  program 
developed  a  test  plan  that  included  a  test  and  evaluation  strategy  and 
schedule. 

Air  Force — BITI  Wired 

The  Air  Force  had  fully  implemented  both  systems  testing  and  integration 
practices  for  the  BITI  Wired  program.  In  this  regard,  the  program  had 
established  testing  roles  and  responsibilities.  The  program  also 
formulated  all  applicable  test  related  plans,  schedules,  and  reports.  For 
example,  the  BITI  Wired  Program  developed  testing  plans  and  schedules 
for  each  base  installation  site. 

Air  Force — JMS  Inc  2 

The  Air  Force  had  fully  implemented  one,  and  partially  implemented  one, 
of  the  systems  testing  and  integration  practices  for  the  JMS  Inc  2 
program.  For  example,  the  program  had  developed  a  Test  and  Evaluation 
Master  Plan  that  provided  the  framework  and  strategy  for  planning  and 
executing  its  testing  and  evaluation  activities.  Although  the  program  had 
established  roles  and  responsibilities  to  manage  testing  and  integration 
activities,  a  chief  developmental  tester  to  oversee  testing  activities  had 
not  been  identified.  Program  officials  stated  that  it  had  been  difficult  to  fill 
this  position  due  to  the  lack  of  qualified  personnel.  Until  this  position  is 
filled,  it  may  be  difficult  to  manage  and  mitigate  risks  during  development, 
verify  that  products  are  compliant  with  operational  requirements,  and 
inform  decision  makers  throughout  the  program’s  life  cycle. 

Army — GCSS-A  Inc  1 

The  Army  had  fully  implemented  both  systems  testing  and  integration 
practices  for  the  GCSS-A  Inc  1  program.  In  this  regard,  the  program  had 
established  and  documented  roles  and  responsibilities  for  all  testing  and 
evaluation  activities,  including  identifying  a  chief  developmental  tester.  In 
addition,  a  GCSS-A  Inc  1  Test  and  Evaluation  Master  Plan  had  been 
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developed,  which  described  the  total  testing  and  evaluation  planning  from 
component  development  through  operational  testing  and  evaluation.  The 
program  also  had  developed  test  plans,  such  as  the  Capstone  Test  Plan 
and  Systems  Engineering  Plan,  which  further  detailed  the  program’s 
testing  and  evaluation  strategy. 

Army— IPPS-A  Inc  2 

The  Army  had  fully  implemented  both  systems  testing  and  integration 
practices  for  the  IPPS-A  Inc  2  program.  In  this  regard,  the  program  had 
established  testing  roles  and  responsibilities  and  formulated  test  plans, 
schedules,  and  reports,  where  applicable.  Specifically,  the  Increment  II 
Test  and  Evaluation  Working  Integrated  Product  Team  Charter  was 
established  to  provide  guidance  in  overseeing  testing  activities.  In 
addition,  a  chief  developmental  tester  had  been  appointed  to  serve  as  the 
focal  point  for  all  testing  and  evaluation  efforts.  Further,  the  IPPS-A  Inc  2 
program  developed  a  Test  and  Evaluation  Master  Plan,  which  described 
the  overall  testing  and  evaluation  plans  from  system  development  through 
operational  testing  and  evaluation  for  all  releases. 


Conclusions 


MAIS  programs  are  important  for  sustaining  DOD’s  key  operations  in 
communications,  business,  and  command  and  control,  and  provide  the 
department  with  access  to  information  to  organize,  plan,  direct,  and 
monitor  mission  operations.  Therefore,  it  is  vital  that  these  programs 
follow  leading  practices  to  ensure  success.  While  a  number  of  leading 
practices  for  risk,  requirements,  and  systems  testing  and  integration  had 
been  fully  or  partially  implemented  by  five  programs  that  we  reviewed  in- 
depth — the  Air  Force’s  AOC  WS  Inc  10.2,  BITI  Wired  and  JMS  Inc  2 
programs;  and  the  Army’s  GCSS-A  Inc  1  and  IPPS-A  Inc  2  programs — 
three  programs  lacked  practices  to  better  manage  the  development  of 
systems.  The  program  office  for  GCSS-A  Inc  1  did  not  have  standard 
operating  procedures  for  defining  thresholds  or  bounds  to  manage  risks, 
thus  putting  at  risk  the  program’s  ability  to  effectively  prioritize,  control, 
and  manage  for  mitigation  and  management  actions.  In  addition,  the 
program  office  for  AOC-WS  Inc  10.2  did  not  have  an  overall  Risk 
Mitigation  Plan  to  guide  the  implementation  of  individual  risk  mitigation 
and  contingency  planning  activities.  An  overall  risk  plan  is  necessary  to 
help  ensure  that  individual  efforts  are  managed  in  a  cohesive  and 
repeatable  way.  Further,  the  program  office  for  JMS  Inc  2  had  not 
assigned  a  chief  developmental  tester,  an  important  role  to  ensure  that 
risks  are  managed  and  mitigated  effectively  throughout  testing  and 
integration.  Unless  this  position  is  filled,  it  will  be  difficult  for  the  program 
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to  effectively  manage  risks  and  verify  compliance  with  certain  operational 
requirements  throughout  the  acquisition. 


Recommendations  for  To  help  improve  the  management  of  DOD's  MAIS  programs,  we  are 

recommending  that  the  Secretary  of  Defense  take  the  following  three 

Executive  Action  actions: 

•  direct  the  Secretary  of  the  Army  to  direct  the  program  manager  for 
GCSS-A  Inc  1  to  establish  standard  operating  procedures  for 
managing  risks  that  include  guidance  for  establishing  thresholds  and 
bounds  for  key  risk  areas. 

•  direct  the  Secretary  of  the  Air  Force  to  direct  the  program  manager  for 
AOC-WS  Inc  10.2  to  develop  an  overall  risk  mitigation  plan  to  guide 
the  implementation  of  individual  risk  mitigation  and  contingency  plan 
activities. 

•  direct  the  Secretary  of  the  Air  Force  to  direct  the  program  manager  for 
JMS  to  appoint  a  chief  developmental  tester  to  oversee  systems 
testing  and  integration  activities. 


DOD  provided  written  comments  on  a  draft  of  this  report,  which  are 
reproduced  in  appendix  III.  In  its  comments,  the  department  agreed  with 
all  three  of  our  recommendations.  Further,  the  department  provided 
evidence  that  it  had  fully  addressed  one  of  the  recommendations  and  it 
described  plans  and  actions  for  addressing  the  remaining  two 
recommendations. 

Regarding  our  recommendation  that  the  GCSS-A  Inc  1  program  establish 
standard  operating  procedures  for  managing  risks,  DOD  provided 
evidence  that  the  Army  had  updated  its  risk  management  plan. 
Specifically,  the  Army’s  GCSS-A  Inc  1  program  manager  amended  the 
Risk  Management  Plan  for  Project  Manager  Army  Enterprise  Systems 
Integration  Program  on  March  1,  2017.  The  plan  now  describes  how  risk 
prioritization  occurs;  it  requires  that  risk  owners  work  with  the  risk 
management  officer  to  define  appropriate  trigger  conditions  in  order  to 
ensure  a  mitigation  plan  can  be  initiated  at  the  appropriate  point.  By 
taking  these  actions  to  implement  this  recommendation,  the  program 
should  be  in  a  better  position  to  more  effectively  manage  risks. 

DOD  also  concurred  with  our  recommendation  to  develop  an  overall  risk 
mitigation  plan  to  guide  the  implementation  of  individual  risk  mitigation 
and  contingency  planning  activities  on  the  AOC  WS  Inc  10.2  program. 


Comments  from  the 
Department  of 
Defense 
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Toward  this  end,  the  Air  Force  reported  that  it  plans  to  restore  the 
program’s  risk  management  practices  to  include  weekly  meetings  to 
review  program  risks  and  publish  a  risk  management  plan  in  April  2017. 
Once  implemented,  these  actions  should  help  the  Air  Force  to  better 
ensure  that  individual  risk  mitigation  and  contingency  plan  activities  are 
conducted  in  a  consistent  and  repeatable  way. 

Further,  DOD  concurred  with  our  recommendation  that  JMS  appoint  a 
chief  developmental  tester  to  oversee  systems  testing  and  integration 
activities.  In  this  regard,  the  Air  Force  reported  that  it  had  identified  a 
qualified  candidate  to  fill  the  Chief  Developmental  Tester  position. 
According  to  the  Air  Force,  the  program  expects  the  new  test  chief  to  be 
in  place  in  July  2017.  Once  the  position  is  filled,  the  program  should  be 
better  able  to  oversee  important  systems  testing  and  integration  activities. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Secretary  of  the  Army;  the 
Secretary  of  the  Air  Force;  and  other  interested  parties.  This  report  also  is 
available  at  no  charge  on  the  GAO  website  at  http://www.gao.gov. 

Should  you  or  your  staffs  have  any  questions  on  information  discussed  in 
this  report,  please  contact  me  at  (202)  512-4456  or  harrisc@gao.gov. 
Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
Affairs  may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made 
major  contributions  to  this  report  are  listed  in  appendix  IV. 


Carol  C.  Harris 
Director 

Information  Technology  Acquisition  Management  Issues 
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The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Ron  Johnson 
Chairman 

The  Honorable  Claire  McCaskill 
Ranking  Member 

Committee  on  Homeland  Security  and  Governmental  Affairs 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 


Page  27 


GAO-17-322  DOD  Major  Automated  Information  Systems 


The  Honorable  Jason  Chaffetz 
Chairman 

The  Honorable  Elijah  Cummings 
Ranking  Member 

Committee  on  Oversight  and  Government  Reform 
House  of  Representatives 

The  Honorable  Kay  Granger 
Chairwoman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


The  National  Defense  Authorization  Act  for  Fiscal  Year  2012  mandated 
that  we  select,  assess,  and  report  on  Department  of  Defense’s  (DOD) 
major  automated  information  system  (MAIS)  programs  annually  through 
March  201 8. 1  This  report  is  the  fifth  in  our  series  of  annual  assessments. 
Our  objectives  were  to:  (1 )  describe  the  extent  to  which  selected  MAIS 
programs  have  changed  their  planned  cost  and  schedule  estimates  and 
met  technical  performance  targets  and  (2)  assess  the  extent  to  which 
selected  MAIS  programs  have  used  leading  IT  acquisition  practices, 
including  requirements  and  risk  management,  and  systems  testing  and 
integration. 

To  address  the  first  objective,  we  used  DOD’s  official  list  of  35  MAIS 
programs  as  of  May  1,  2016,  to  establish  a  basis  for  selecting  programs. 
We  selected  18  of  35  MAIS  programs  based  on  our  criteria  that  programs 
must  be  unclassified  and  have  a  first  acquisition  program  baseline  that 
could  be  used  as  a  reference  point  for  evaluating  cost,  schedule,  and 
technical  performance  characteristics. 

For  each  selected  program,  we  asked  relevant  DOD  officials  to  complete 
a  data  collection  instrument  with  questions  on  estimated  cost,  schedule, 
and  technical  performance  goals,  including  the  latest  program  status  in 
meeting  those  estimated  goals.  Using  the  responses  from  the  data 
collection  instrument,  we  then  compared  each  program’s  first  acquisition 
program  baseline  cost  estimate  to  the  latest  estimate  to  determine  the 
extent  to  which  planned  program  costs  had  changed.  Similarly,  to 
determine  the  extent  to  which  these  programs  had  changed  their  planned 
schedule  estimates,  we  compared  each  program’s  first  acquisition 
program  baseline  schedule  estimate  to  the  latest  schedule. 

To  determine  whether  the  selected  programs  met  their  performance 
targets,  we  reviewed  the  most  recent  program  management  reports  and 
examined  each  program’s  self-identified  system  performance  targets  and 
the  status  of  their  progress.  We  also  reviewed  additional  information  on 
each  program’s  cost,  schedule,  and  performance,  including  program 
documentation,  such  as  DOD’s  MAIS  annual  and  quarterly  reports, 
acquisition  program  baselines,  system  test  reports,  and  our  prior  reports. 
We  then  aggregated  and  summarized  the  results  of  these  analyses 
across  the  programs,  and  developed  individual  profiles  for  each  program. 


Vub.  L.  No.  112-81  §  1078  (Dec.  31, 2011). 
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To  address  the  second  objective,  we  started  with  the  list  of  18  MAIS 
programs  that  were  selected  specifically  for  the  first  objective.  From  this 
list,  we  identified  programs  for  a  more  in-depth  review  based  on  two 
criteria:  the  programs  selected  must  represent  at  least  two  military 
services,  and  the  programs  must  have  not  been  evaluated  by  us  within 
the  last  year.2  Using  this  criteria,  we  identified  5  programs  from  two 
military  services:  the  Army’s  Global  Combat  Supply  System  Increment  1 
(GCSS-A  Inc  1)  and  Integrated  Personnel  and  Pay  System  Increment  2 
(IPPS-A  Inc  2);  and  the  Air  Force’s  Air  and  Space  Operations  Center- 
Weapon  System  Increment  10.2  (AOC-WS  Inc  10.2),  Base  Information 
Transport  Infrastructure  Wired  (BITI  Wired),  and  Joint  Space  Operations 
Center,  Mission  System  Increment  2  (JMS  Inc  2). 

Based  on  prior  work  evaluating  MAIS  programs,  we  identified  three 
management  practice  areas — requirements  management,  risk 
management,  and  systems  testing  and  integration  that  are  applicable  to 
these  programs.  For  evaluating  how  the  selected  five  programs  applied 
practices  in  the  risk  and  requirements  management  areas,  we  identified 
key  practices  related  to  these  areas  from  the  Software  Engineering 
Institute’s  Capability  Maturity  Model®  Integration  for  Acquisition  (CMMI- 
ACQ).3 

Specifically,  for  requirements  management,  we  evaluated  each  of  the  five 
programs  against  the  following  5  practices: 

•  develop  an  understanding  with  the  requirements  providers  of  the 
meaning  of  the  requirements, 

•  obtain  commitment  to  requirements  from  project  participants, 

•  manage  changes  to  requirements  as  they  evolve  during  the  project, 

•  maintain  bidirectional  traceability  among  requirements  and  work,  and 

•  ensure  project  plans  and  work  products  remain  aligned  with 
requirements. 

In  doing  so,  we  analyzed  program  requirements  documentation  against 
the  practices.  This  documentation  included  requirements  management 
plans,  traceability  matrices,  requirements  change  forms,  technical 


2GAO- 16-336 

3Software  Engineering  Institute,  Capability  Maturity  Model®  Integration  for  Acquisition 
(CMMI-ACQ),  Version  1.3  (Pittsburgh,  Pa.:  November  2010). 
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performance  assessments,  and  requirements  board  meeting  minutes.  We 
also  interviewed  program  officials  to  obtain  additional  information  about 
their  requirements  management  practices. 

For  the  risk  management  area,  we  evaluated  each  of  the  five  programs 
against  the  following  7  practices: 

•  determine  risk  sources  and  categories; 

•  define  parameters  used  to  analyze  and  categorize  risks  and  to  control 
the  risk  management  effort; 

•  establish  and  maintain  the  strategy  to  be  used  for  risk  management; 

•  identify  and  document  risks; 

•  evaluate  and  categorize  each  identified  risk  using  defined  risk 
categories  and  parameters,  and  determine  its  relative  priority; 

•  develop  a  risk  mitigation  plan  in  accordance  with  the  risk  management 
strategy;  and 

•  monitor  the  status  of  each  risk  periodically  and  implement  the  risk 
mitigation  plan  as  appropriate. 

In  doing  so,  we  analyzed  each  program’s  key  documents,  such  as  risk 
management  plans,  risk  charts,  and  other  artifacts,  to  determine  the 
extent  to  which  these  practices  had  been  applied.  We  also  interviewed 
program  officials  to  obtain  additional  information  about  their  risk 
management  practices. 

For  the  systems  testing  and  integration  area,  we  evaluated  each  of  the 
five  programs  against  the  following  2  practices: 

•  establish  roles  and  responsibilities  to  manage  testing  and  integration 
activities  and 

•  develop  test-related  plans,  schedules,  and  reporting. 

In  this  regard,  we  evaluated  the  test  manager’s  roles  and  responsibilities, 
as  well  as  test  plans,  schedules,  and  test  reports  to  determine  the  extent 
that  these  practices  had  been  applied.  We  also  conducted  interviews  with 
program  officials  to  obtain  additional  information  about  their  systems  test 
and  integration  practices. 

Further,  in  determining  the  extent  to  which  each  of  the  five  MAIS 
programs  had  implemented  the  three  practice  areas — requirements 
management,  risk  management,  and  systems  testing  and  integration — we 
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rated  a  practice  as  “fully  implemented”  when  all  of  the  practices  for  the 
area  had  been  implemented.  Similarly,  when  a  program  had  implemented 
at  least  one  of  the  practices  within  the  area,  we  rated  the  practice  as 
“partially  implemented”  to  indicate  the  practice  had  been  partially 
implemented.  For  a  MAIS  program  that  did  not  conduct  activities  for  any 
of  the  practices  because  the  program  had  been  initiated  after  a  limited 
production  decision  point  (post-Milestone  C),  we  assessed  the  practice  as 
“not  applicable”  because  such  programs  do  not  apply  practices  to 
develop,  manage  requirements,  or  test  the  systems. 

To  assess  the  reliability  of  the  data  that  we  used  to  support  the  findings  in 
this  report,  we  corroborated  program  office  responses  with  relevant 
program  documentation  and  interviews  with  agency  officials.  We  found  no 
data  reliability  issues  and  determined  that  the  data  used  in  this  report 
were  sufficiently  reliable  for  our  reporting  purposes.  We  have  also  made 
appropriate  attribution  indicating  the  sources  of  the  data. 

We  conducted  this  performance  audit  from  April  2016  to  March  2017  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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This  section  contains  profiles  of  the  18  selected  MAIS  programs  that  we 
analyzed  to  determine  whether  they  had  changed  their  planned  cost  and 
schedule  estimates  and  met  performance  measures.  Each  profile 
presents  data  on  the  program’s  purpose  and  status,  its  latest  cost  and 
schedule  estimates  compared  to  the  first  acquisition  program  baseline 
and  system  performance  data.  The  first  page  of  each  two-page  profile 
provides  (1)  a  description  of  the  program’s  purpose,  (2)  essential  details, 
such  as  the  responsible  DOD  component,  program  owner,  prime 
contractor,  the  total  number  of  active  contractors  supporting  the  program; 
and  (3)  program  costs  (in  then-year  dollars),1  comparing  the  program’s 
latest  life-cycle  cost  estimate  (separated  into  acquisition  and  operations 
and  maintenance  costs)  to  its  first  acquisition  program  baseline 
(subsequent  acquisition  program  baselines  that  may  have  been 
established  are  not  identified). 

The  profile  also  provides  information  on  the  program’s  status,  costs, 
schedule,  and  performance. 

•  In  the  program  status  section,  we  discuss  recent  milestones  achieved 
and  acquisition  events. 

•  In  the  cost  section,  we  identify  the  extent  to  which  the  program’s  life- 
cycle  cost  estimate  has  changed  from  its  first  acquisition  program 
baseline,  as  well  as  any  notable  causes  for  any  changes. 

•  In  the  schedule  section,  we  discuss  the  extent  to  which  the  program’s 
schedule  has  changed  from  its  first  acquisition  program  baseline  and 
causes  for  significant  changes. 

In  the  performance  section,  we  identify  the  extent  to  which  each  program 
has  met  its  established  measures.  These  performance  ratings  represent 
point-in-time  assessments  as  reported  by  the  program.  System 
performance  targets  are  rated  as  “met”  when  the  program  fully  met  all  of 
its  key  performance  parameters.  System  performance  was  rated  as  “did 
not  fully  meet”  when  a  program  did  not  pass  system  testing.  System 
performance  was  rated  as  “not  yet  applicable”  when  the  system  had  not 
yet  been  tested  because  the  program  was  not  yet  at  the  appropriate 
acquisition  phase. 


^hen-year  dollars  are  those  dollars  that  include  the  effects  of  inflation  or  escalation 
and/or  reflect  the  price  levels  expected  to  prevail  during  the  year  at  issue. 
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Air  Force — Air  and  Space 
Operations  Center- 
Weapon  System 
Increment  10.2  (AOC-WS 
Inc  10.2) 


Program  description 

The  AOC-WS  Inc  10.2  program  is  expected  to  provide  system  hardware, 
software,  technical  documents,  and  technology  refresh  to  ensure  it 
remains  a  viable  weapons  system.  The  program  will  enable  personnel  at 
selected  air  and  space  operations  centers  to  better  plan,  execute,  and 
assess  theater-wide  air  and  space  operations.  It  will  replace  the  currently 
fielded  AOC  10.1  system  and  provide  additional  capabilities,  such  as 
dynamic  planning  and  execution;  data  management;  information 
assurance;  predictive  battlespace  awareness;  and  airspace  management. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Air  Force 

Program  owner:  Air  Combat  Command 

Prime  contractor:  Northrop  Grumman,  Mission  Systems 

Total  number  of  contractors  (as  of  May  2016):  10 


Table  4:  Air  and  Space  Operations  Center-Weapon  System  Increment  10.2  Program 
costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
October  2013 

Latest 
estimate 
June  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$374.7 

$604.2 

Procurement 

88.0 

101.8 

Total  acquisition  cost 

462.7 

706.0 

Life-cycle  cost  estimate 

Operations  and  support 

5,122.5 

4,936.0 

Total  life-cycle  cost  estimate 

$5,585.2 

$5,642.0 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  June  201 6,  the  AOC-WS  Inc  1 0.2  program  was  in  the  engineering 
and  manufacturing  development  phase  of  acquisition  in  the  Defense 
Acquisition  Management  System.  In  March  2016,  the  program  declared  a 
critical  change  because  of  significant  cost  and  schedule  growth  due  to 
schedule  and  technical  difficulties  in  reaching  Milestone  C,  which  is  the 
key  juncture  for  approval  to  move  into  limited  production.  According  to  the 
program  office,  major  factors  contributing  to  the  cost  and  schedule  delays 
were  the  prime  contractor  (1)  underestimating  the  software  development 
effort;  (2)  insufficiently  allocating  manpower;  and  (3)  underestimating  the 
end-to-end  testing  capability. 

Exceeded  planned  cost  estimate 

As  of  June  2016,  the  AOC-WS  Inc  10.2  program’s  total  life-cycle  cost  was 
$5,642.0,  an  increase  of  $56.8  million  (1  percent)  from  the  original 
estimate  of  $5,585.2.  The  estimated  total  acquisition  cost  is  $706.0 
million,  an  increase  of  $243.3  million  (53  percent)  from  the  original 
estimate  of  $462.7  million. 

Exceeded  planned  schedule  estimate 

The  AOC-WS  Inc  10.2  program  experienced  significant  schedule  delays 
since  establishing  the  original  schedule  in  2007.  Specifically,  Milestone  C 
was  estimated  to  be  completed  on  July  2015,  but,  as  of  September  2016, 
the  program  had  not  reached  the  milestone.  According  to  program 
officials,  the  program  was  undergoing  a  critical  change  that  will  result  in  a 
3-year  schedule  delay. 

Not  yet  applicable 

The  AOC-WS  Inc  10.2  program  was  still  early  in  development  and  system 
performance  data  was  not  available  at  the  time  of  our  review. 
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Air  Force — Base 
Information  Transport 
Infrastructure  Wired 
(BITI  Wired) 


Program  description 

The  BITI  Wired  program  is  to  upgrade  the  enterprise  network  and  provide 
the  core  cyber  infrastructure.  The  upgrade  includes  wired  and  wireless 
systems,  and  network  cables  and  servers  for  more  than  150  active  duty, 
Air  Force  Reserve,  and  Air  National  Guard  bases.  The  program  is 
expected  to  update  the  fixed  local  area  network  and  all  necessary 
information  transport  infrastructure  to  support  current  and  future 
communications  needs  of  the  Air  Force  and  Joint  Command  warfighter. 
This  includes  enabling  base  level  data,  voice,  video,  imagery,  sensor,  and 
telemetry  requirements. 

Program  essentials  (as  of  October  2016) 

DOD  component:  Air  Force 

Program  owner:  Air  Force  Space  Command  (AFSPC) 

Prime  contractors:  General  Dynamics,  Harris  Corp,  Lockheed  Martin, 
Telos  Corp,  Three  Wire  Systems,  and  Red  River  for  the  procurement 
contracts  and  Affigent  and  ThunderCat  for  the  operations  and  support 
contracts 

Total  number  of  contractors  (as  of  May  2016):  8 


Table  5:  Base  Information  Transport  Infrastructure  Wired  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

June  2010 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$0.0 

$0.0 

Procurement 

1,417.7 

1,177.2 

Total  acquisition  cost 

1,417.7 

$1,177.2 

Life-cycle  cost  estimate 

Operations  and  support 

2,372.2 

1,146.5 

Total  life-cycle  cost  estimate 

$3,789.9 

$2,323.7 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

The  BITI-Wired  program  achieved  a  full  deployment  decision  on  February 
18,  2014,  and  expects  to  reach  full  deployment  in  September  2017.  The 
program  completed  167  out  of  178  turnkey  network  upgrades  and 
expects  to  remain  within  the  cost,  schedule,  and  performance  thresholds. 

Decreased  cost  estimate 

As  of  July  2016,  the  BITI  Wired  program’s  total  life-cycle  cost  estimate 
was  $2,323.7  million,  a  decrease  of  $1,466.2  million  (-39  percent)  from 
the  original  estimate  of  $3,789.9  million.  The  decrease  was  due  to  more 
accurate  cost  estimates  being  provided  by  vendors.  Specifically,  initial 
estimates  included  large  risk  factors  because,  at  the  time,  the  government 
could  not  provide  vendors  a  definitive  list  of  items.  As  a  result,  vendors’ 
earlier  estimates  reflected  the  potential  for  risks  in  their  estimates.  The 
new  list  of  refined  equipment  resulted  in  reduced  costs.  In  addition,  while 
officials  cited  insufficient  human  resources  as  a  potential  risk  to  cost  as 
the  program  moves  forward,  they  fully  expect  to  stay  within  the  cost 
estimates. 

Met  planned  schedule  estimate 

The  BITI  Wired  program’s  full  deployment  decision  was  2  months’  behind 
schedule.  However,  program  officials  report  they  are  on  track  to  meet  full 
deployment  in  September  2017. 

Did  not  fully  meet  all  of  the  system  performance  targets 

The  BITI-Wired  program  had  partially  implemented  all  4  of  performance 
targets — net  ready,  availability,  support,  and  reliability.  According  to 
program  officials,  installation  of  equipment  had  been  installed  at  167  of 
the  178  sites.  The  program  expects  to  report  all  4  performance  targets  as 
fully  met  in  September  2017,  the  full  deployment  date,  when  the 
equipment  has  been  installed  at  all  sites. 
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Navy — Consolidated  Afloat 
Networks  and  Enterprise 
Services  (CANES) 


Program  description 

The  CANES  program  is  expected  to  consolidate  and  standardize  the 
Navy’s  existing  network  infrastructures  and  services  by  reducing  and 
eliminating  existing  standalone  afloat  (i.e.,  surface  ships  and  submarines) 
networks,  providing  a  technology  platform  that  can  rapidly  adjust  to 
changing  warfighting  requirements,  and  reducing  the  hardware  footprint 
on  259  afloat  and  maritime  operations  center  platforms. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Navy 
Program  owner:  C4I 

Prime  contractors:  Northrop  Grumman,  Serco,  BAE  Systems,  DRS  Laurel 
Technologies,  General  Dynamics,  CGI  Federal,  Global  Technical 
Systems 

Total  number  of  contractors  (as  of  May  2016):  82 


Table  6:  Consolidated  Afloat  Networks  and  Enterprise  Services  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
January  2011 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and  evaluation 

$501.1 

$631.3 

Procurement 

3,476.1 

3,746.7 

Total  acquisition  cost 

$3,977.2 

$4,378.0 

Life-cycle  cost  estimate 

Operations  and  support 

8,763.7 

7,532.6 

Total  life-cycle  cost  estimate 

$12,740.9 

$11,910.6 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

The  CANES  program  had  moved  past  the  full  deployment  milestone.  The 
program  is  expected  to  have  all  systems  fully  fielded  by  September  2024. 
According  to  program  officials,  risks  remain  that  could  result  in  delaying 
the  fielding  of  CANES,  thereby  increasing  risk  to  the  program’s  ability  to 
achieve  full  deployment  on  schedule. 

Decreased  planned  cost  estimate 

As  of  July  2016,  the  CANES  program’s  total  life-cycle  cost  estimate  was 
$11,91 0.6  million,  a  decrease  of  $830.3  million  (-7  percent)  from  the 
original  estimate  of  $12,740.9  million.  In  February  2016,  in  response  to  a 
full  deployment  decision,  the  life-cycle  cost  estimate  was  updated  to 
reflect  reductions  in  procurement  costs,  the  program’s  technology  refresh 
schedule,  and  manpower  costs. 

Exceeded  planned  schedule  estimate 

The  CANES  program  experienced  a  significant  delay  in  reaching  a  full 
deployment  decision.  In  June  2014,  the  program  slipped  the  objective 
date  for  full  deployment  decision  2  years,  from  December  2013  to 
December  2015.  A  full  deployment  decision  was  achieved  in  October 
2015,  and  full  deployment  is  estimated  for  September  2024.  According  to 
program  officials,  an  unstable  funding  environment  is  the  largest  risk  to 
achieving  full  deployment. 

Did  not  fully  meet  all  of  the  system  performance  targets 

The  CANES  program  had  not  met  all  9  of  the  performance  targets. 
Specifically,  of  the  9  performance  targets,  8  have  been  met,  while  1 
performance  target  was  partially  met — the  net  ready  performance  target. 
According  to  the  program  office,  this  performance  target  was  only  partially 
met  because  testing  for  afloat  core  services  has  been  deferred. 
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Navy — Common  Aviation 
Command  and  Control 
System  Increment  1 
(CAC2S  Inc  1) 


Program  description 

The  CAC2S  Inc  1  program  is  expected  to  provide  an  integrated  and 
coordinated  modernization  effort  for  the  equipment  of  the  Marine  Air 
Command  and  Control  System.  The  system  is  expected  to  provide 
enhanced  capability  for  three  air  defense  centers  to  support  aviation 
employment  in  joint,  combined,  and  coalition  operations.  It  will  provide 
tactical  situational  display,  information  management,  sensor  and  data  link 
interface,  and  operational  facilities  for  planning  and  execution  of  Marine 
Aviation  missions. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Navy 

Program  owner:  USMC,  Program  Executive  Officer,  Land  Systems 
Prime  contractors:  General  Dynamics,  C4  Systems,  Inc. 

Total  number  of  contractors  (as  of  May  2016):  5 


Table  7:  Common  Aviation  Command  and  Control  System  Increment  1  Program 
costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
August  2000 

Latest 
estimate 
March  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$559.3 

Procurement 

322.8 

Operations  and  maintenance 

Total  acquisition  cost 

$173.4 

$882.1 

Life-cycle  cost  estimate 

Operations  and  support 

173.6 

1,090.6 

Total  life-cycle  cost  estimate 

$347.0 

$1,972.7 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  Phase  1  of  the  CAC2S  Inc  1  program  was  in  the 
operations  and  sustainment  acquisition  phase  and  Phase  2  was  in  the 
production  and  deployment  phase.  The  program  has  been  rebaselined 
once  following  a  critical  change  that  occurred  in  2009  due  to  the 
program’s  inability  to  meet  the  initial  operational  capability  date. 

Exceeded  planned  cost  estimate 

As  of  March  2016,  the  CAC2S  Inc  1  program’s  total  estimated  life-cycle 
estimate  was  $1 ,972.7  million,  an  increase  of  $1 ,625.7  million  (469 
percent)  from  the  original  life-cycle  estimate  of  $347  million.  Similarly,  the 
program’s  current  total  acquisition  estimate  is  $882  million,  an  increase  of 
$708.7  million  (409  percent)  from  the  original  total  acquisition  estimate  of 
$173.4  million.  As  we  reported  in  March  201 6, 2  factors  that  contributed  to 
the  cost  increase  were  early  challenges  in  estimating  costs  due  to 
program  scope  growth  and  need  for  restructuring.  Program  officials  noted 
funding  instability  as  a  potential  area  of  risk  in  meeting  cost  estimates. 

Exceeded  planned  schedule  estimate 

As  of  October  2015,  the  CAC2S  Inc  1  program  had  a  13-year,  9-month 
slippage  in  their  full  deployment  date  due  to  a  major  change  in  project 
scope  and  restructuring  of  the  project. 

Met  all  of  the  system  performance  targets 

CAC2S  Inc.  1  had  met  both  of  the  technical  performance  targets  which 
were  (1)  net  ready,  and  (2)  data  fusion. 
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Defense  Logistics 
Agency — Defense 
Agencies  Initiative 
Increment  2  (DAI  Inc  2) 


Program  description 

The  DAI  Inc  2  program’s  mission  is  expected  to  deliver  auditable, 
accurate,  timely,  authoritative  financial  data  to  support  DOD’s  goal  of 
standardizing  financial  management  practices,  improving  financial 
decision  support,  and  supporting  audit  readiness.  Defense  agencies  use 
more  than  ten  different  non-compliant  financial  management  systems 
supporting  diverse  operational  functions  and  the  warfighter  in  decision 
making  and  financial  reporting. 

Program  essentials  (as  of  October  2016) 

DOD  component:  Defense  Logistics  Agency 
Program  owner:  OSD  (C) 

Prime  contractor:  Credence  Management  Solutions,  LLC;  TeraThink 
Corp;  IBM;  CACI,  Inc;  Northrop  Grumman  Systems  Corp;  CACI  ISS,  Inc; 
Amyx,  Inc 

Total  number  of  contractors  (as  of  May  2016):  25 


Table  8:  Defense  Logistics  Agency- 
Program  costs 

(then-year  dollars  in  millions) 

-Defense  Agencies  Initiative  Increment  2 

First  acquisition 
program  baseline 

July  2014 

Latest 
estimate 
March  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$155.3 

$157.9 

Operation  and  maintenance 

27.0 

27.0 

Total  acquisition  cost 

$182.3 

$184.9 

Life-cycle  cost  estimate 

Operations  and  support 

683.9 

683.9 

Total  life-cycle  cost  estimate 

$866.2 

$868.8 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  DAI  Inc  2  program  was  in  the  limited  deployment 
phase  of  the  business  capability  life  cycle.  According  to  a  2016  MAIS 
annual  report,  the  program  was  on  track  to  remain  within  the  schedule, 
cost,  and  performance  thresholds  identified  in  the  original  estimate. 
According  to  program  officials,  the  program  was  rebaselined  to  adjust 
release  2  objective  and  threshold  dates  for  limited  fielding  to 
accommodate  the  time  frame  for  resolving  a  bid  protest.  Notwithstanding 
this  additional  time  for  the  bid  protest,  the  program  achieved  the  original 
acquisition  program  baseline  objective  date  for  this  event. 

Exceeded  planned  cost  estimate 

As  of  March  2016,  the  DAI  Inc  2  program’s  total  life-cycle  estimate  was 
$868.8  million,  an  increase  of  $2.6  million  (<1  percent)  from  the  original 
life-cycle  cost  estimate  of  $866.2  million.  Similarly,  the  program’s  current 
total  acquisition  estimate  was  $184.9  million,  an  increase  of  $2.6  million 
(1  percent)  from  the  original  acquisition  estimate  of  $182.3  million. 
However,  these  cost  increases  were  within  the  planned  cost  objective 
estimates.  Officials  cited  funding  instability  as  a  potential  risk  to  cost  as 
the  program  moves  forward. 

Met  planned  schedule  estimate 

The  DAI  Inc  2  program  had  met  the  first  acquisition  program  baseline 
objective  schedule  estimate  for  limited  fielding  in  September  2015. 
Officials  noted  they  were  on  schedule  to  stay  within  the  threshold  date  of 
April  2017  for  a  full  deployment  decision. 

Did  not  fully  meet  all  of  the  system  performance  targets 

According  to  the  2016  MAIS  annual  report,  the  DAI  Inc  2  program  had  not 
met  any  of  the  five  of  the  technical  performance  targets  which  are  (1) 
financial  system  performance,  (2)  business  enterprise  architecture 
compliance,  (3)  support  net  centric  DOD  military  operations,  (4)  managed 
in  network,  and  (5)  effectively  exchange  information. 
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Department  of  Defense — 
Healthcare  Management 
System  Modernization 
(DHMSM) 


Program  description 

The  DHMSM  Program  is  expected  to  provide  a  configurable  and  scalable 
modernized  electronic  Health  Record  System.  It  will  replace  DOD’s 
legacy  healthcare  systems  including  the  Armed  Force’s  Health 
Longitudinal  Technology  Application,  Composite  Health  Care  System 
(inpatient),  and  most  components  of  the  Theater  Medical  Information 
Program-Joint  program,  with  a  commercial  Off-The-Shelf  electronic  health 
record  system.  It  is  expected  to  address  the  current  state  of  the  Military 
Health  System,  where  multiple  legacy  systems  and  data  stores, 
developed  over  decades,  are  in  need  of  modernization. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Defense  Health  Agency 

Program  owner:  PEO  Defense  Healthcare  Management  Systems 

Prime  contractor:  Leidos 

Total  number  of  contractors  (as  of  May  2016):  35 


Table  9:  Department  of  Defense  Healthcare  Management  System  Modernization 
Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

May  2016 

Latest 

estimate 

N/A 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$881.9 

N/A 

Procurement 

2372.7 

N/A 

Operations  and  maintenance 

480.5 

N/A 

Total  acquisition  cost 

$3,735.1 

N/A 

Life-cycle  cost  estimate 

Operations  and  support 

4,929.5 

N/A 

Total  life-cycle  cost  estimate 

$8,664.6 

N/A 

Source:  DOD  data  |  GAO-17-322 


Cost,  schedule,  and  performance  summary 

Not  applicable:  The  program  obtained  its  first  baseline  on  May  3,  2016. 
Due  to  the  recent  establishment  of  the  program’s  first  baseline,  a  history 
of  cost,  schedule,  and  technical  performance  has  not  been  established. 
Therefore,  the  cost,  schedule,  and  technical  performance  is  not 
discussed. 
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Air  Force — Defense 
Enterprise  Accounting 
and  Management 
System-Increment  1 
(DEAMS  Inc  1) 


Program  description 

The  DEAMS  Inc  1  program  is  expected  to  provide  the  Air  Force  with  an 
entire  spectrum  of  financial  management  capabilities,  including 
collections;  commitments  and  obligations;  cost  accounting;  general 
ledger;  funds  control;  receipts  and  acceptance;  accounts  payable  and 
disbursement;  billing;  and  financial  reporting.  The  DEAMS  program  is 
also  intended  to  be  a  key  component  of  DOD’s  solution  for  achieving 
fully-auditable  financial  statements  by  September  30,  2017,  as  required 
by  the  National  Defense  Authorization  Act  for  Fiscal  Year  2010. 

Program  essentials  (as  of  October  2016) 

DOD  component:  Air  Force 
Program  owner:  Air  Force  Financial  Management 
Prime  contractors:  Accenture  and  Kearney 
Total  number  of  contractors  (as  of  May  2016):  29 


Table  10:  Defense  Enterprise  Accounting  and  Management  System-Increment  1 
Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
February  2012 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$591.1 

$560.7 

Procurement 

50.0 

31.4 

Total  acquisition  cost  estimate 

$641.1 

$592.1 

Life-cycle  cost  estimate 

Operations  and  support 

$640.6 

$813.7 

Other 

152.6 

157.9 

Total  life-cycle  cost  estimate 

$1,434.3 

$1,563.7 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  June  2016,  the  DEAMS  Inc  1  program  was  in  the  limited 
deployment  phase.  In  March  2016,  the  program  was  approved  for  limited 
deployments  at  smaller  major  commands.  Since  then,  the  program  has 
undergone  a  Critical  Change  due  to  schedule  slippage  of  the  threshold 
full  deployment  date  of  more  than  a  year.  However,  the  Critical  Change 
report  was  not  completed  at  the  time  of  our  review.  Specifically,  the 
DEAMS  Inc  1  program  was  not  demonstrating  enough  progress  towards 
operational  effectiveness,  suitability,  and  survivability  to  reach  the  full 
deployment  date  within  one  year  of  the  milestone  objective.  In  addition, 
the  program  office  identified  requirements  management  and  technical 
risks  as  areas  that  pose  risks  to  successfully  meeting  schedule 
milestones. 

Exceeded  planned  cost  estimate 

As  of  July  2016,  the  DEAMS  Inc  1  program’s  total  life-cycle  estimate  was 
$1 ,563.7,  an  increase  of  $1 24.9  million  (9  percent)  from  the  original  life- 
cycle  estimate  of  $1,434.3  million.  In  addition,  the  program’s  cost 
estimate  decreased  by  $49.0  million  (-8  percent)  when  compared  to  the 
original  total  acquisition  estimate  of  $641 .1 . 

Exceeded  planned  schedule  estimate 

The  DEAMS  Inc  1  program  was  undergoing  a  critical  change  due  to 
schedule  slippages.  The  full  deployment  date  has  been  exceeded  by 
more  than  a  year.  However,  a  Critical  Change  report  had  not  yet  been 
issued  at  the  time  of  our  review. 

Did  not  fully  meet  all  of  the  system  performance  targets 

The  DEAMS  Inc  1  program  had  partially  met  the  system  performance 
targets.  Specifically,  6  of  the  9  performance  targets  have  been  met,  while 
3  have  not. 
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Global  Combat  Support 
System — Army  Increment 
1  (GCSS-A  Inc  1) 


Program  description 

The  GCSS-A  Inc  1  program  is  a  commercial  off-the-shelf  enterprise 
resources  planning  system  that  is  expected  to  provide  users  with  supply, 
maintenance,  property,  integrated  materiel  management,  and 
management  functionality  and  support.  It  uses  non-developmental  items 
made  up  of  software  and  server  hardware  and  has  two  components: 
GCSS-Army,  which  provides  logistics  and  financial  operations,  and  the 
Army  Enterprise  System  Integration  Program,  which  is  expected  to 
provide  enterprise  hub  services,  master  data  management,  and  business 
intelligence. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Army 

Program  owner:  Assistant  Secretary  of  the  Army,  Acquisition,  Logistics 
and  Technology 

Prime  contractor:  Northrop  Grumman 

Total  number  of  contractors  (as  of  May  2016):  Northrop  Grumman 


Table  11:  Global  Combat  Support  System- 

(then-year  dollars  in  millions) 

—Army  Increment  1  Program  costs 

First  acquisition 
program  baseline 
April  2013 

Latest 
estimate 
May  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$997.4 

$1,059.7 

Procurement 

834.5 

873.0 

Total  acquisition  cost 

$1,831.9 

$1,932.7 

Life-cycle  cost  estimate 

Operations  and  support 

2,137.0 

1,977.7 

Total  life-cycle  cost  estimate 

$3,968.9 

$3,910.4 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  GCSS-A  Inc  1  program  was  in  the  production  and 
deployment  phase  of  the  Defense  Acquisition  Management  System. 
According  to  the  2016  Annual  Report,  the  program  was  on  track  to  remain 
within  schedule,  cost,  and  performance  thresholds.  However,  the  GCSS- 
A  Inc  1  program  realized  a  decrease  of  $16  million  in  funding  that  could 
potentially  cause,  in  part,  a  six-month  slip  in  fielding  with  a  completion 
date  of  March  2018. 

Decreased  planned  cost  estimate 

As  of  May  2016,  the  GCSS-A  Inc  1  program’s  total  life-cycle  estimate  was 
$3,910.4  million,  a  decrease  of  $58.5  million  (-1.5  percent)  from  the 
original  life-cycle  estimate  of  $3,968.9  million.  The  current  total  acquisition 
estimate  was  $1,932.7  million,  an  increase  of  $100.8  million  (5.5  percent) 
from  the  original  total  acquisition  estimate  of  $1,831.9  million. 

Exceeded  planned  schedule  estimate 

The  GCSS-A  Inc  1  program  reached  the  full  deployment  decision  in 
December  2012,  10  months  after  the  objective  date  of  February  2012, 
and  4  months  after  the  threshold  date  of  August  2012.  This  represents  a 
significant  change  from  the  original  estimate  of  February  2012.  The 
program  office  cited  the  potential  for  a  6-month  slip  in  schedule  to  meet 
the  full  deployment  date  of  September  2017.  According  to  a  MAIS 
summary  report,  the  program  conducted  a  trade-off  analysis  to  minimize 
a  6-month  slip  in  fielding,  as  a  result  of  a  $16  million  decrement  in  2016 
funding.  Other  challenges  in  meeting  schedule  include  requirements 
management,  insufficient  human  resources,  and  the  possibility  for  Army 
schedule  changes.  However,  the  program  reports  they  are  on  schedule 
for  full  deployment  prior  to  the  threshold  schedule  date  of  March  2018. 

Met  all  of  the  system  performance  targets 

The  GCSS-A  Inc  1  program  had  met  both  of  the  systems  performance 
targets  which  were  (1)  net  ready,  and  (2)  mission  critical  functions. 
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Marine  Corps — Global 
Combat  Support  System- 
Marine  Corps  Logistics 
Chain  Management 
Increment  1  (GCSS-MC 
LCM  Inc  1) 


Program  description 

The  GCSS-MC  LCM  Inc  1  program  is  a  portfolio  of  systems  that  is 
expected  to  support  logistics  elements  of  command  and  control,  joint 
logistics  interoperability,  and  secure  access  to  and  visibility  of  logistics 
data.  As  a  program  within  GCSS-MC,  Increment  1  is  based  on  the 
implementation  of  Oracle  e-Business  Suite  1 1  i  as  the  core  software 
package.  It  is  expected  to  provide  the  foundation  for  all  future  Marine 
Corps  logistics  systems  modernization. 


Program  essentials  (as  of  October  2016) 

DOD  components:  Department  of  the  Navy,  United  States  Marine  Corps 
Program  owner:  Deputy  Commandant  Installations  and  Logistics 
Headquarters  Marine  Corp 

Prime  contractor:  N/A,  managed  by  the  federal  government 
Total  number  of  contractors  (as  of  May  2016):  Not  applicable 


Table  12:  Global  Combat  Support  System-Marine  Corps  Logistics  Chain 

Management  Increment  1  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

June  2007 

Latest 
estimate 
March  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$102.3 

$307.9 

Procurement 

52.5 

88.3 

Operations  and  maintenance 

39.6 

160.5 

Total  acquisition  cost 

$194.4 

$556.7 

Life-cycle  cost  estimate 

Operations  and  support 

306.6 

1,265.0 

Total  life-cycle  cost  estimate 

$461.4 

$1,821.7 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  GCSS-MC  LCM  Inc  1  program  had  been  fully 
implemented  and  reported  as  a  closed  acquisition  program. 

Exceeded  planned  cost  estimate 

As  of  March  2016,  the  GCSS-MC  LCM  Inc  1  program’s  total  life  cycle 
cost  estimate  was  $1 ,821 .70  million,  an  increase  of  $1 ,360.3  million  (295 
percent)  from  the  original  total  life  cycle  cost  estimate  of  $461.4  million. 
The  GCSS-MC  LCM  Inc  1  program’s  current  total  acquisition  cost 
estimate  was  $556.7  million,  an  increase  of  $362.3  million  (186  percent) 
from  the  original  estimate  of  $194.4  million. 

Exceeded  planned  schedule  estimate 

The  GCSS-MC  LCM  Inc  1  program  had  a  significant  change  in  reaching 
full  deployment.  The  program  reached  the  full  deployment  date  in 
December  2015.  When  compared  to  the  original  date  of  May  2010  for  full 
operational  capability  (equivalent  is  full  deployment),  this  resulted  in  a  5 
year  and  7  month  delay.  The  program  is  fully  implemented. 

Met  all  of  the  system  performance  targets 

As  of  March  2016,  the  GCSS-MC  Inc  1  program  had  met  both  of  the 
performance  targets  which  were  (1)  Net  ready,  and  (2)  Visibility 
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Defense  Health  Agency — 
Integrated  Electronic 
Health  Record  Increment 
1  (iEHR  Inc  1) 


Program  description 

The  iEHR  Inc  1  program  is  expected  to  deliver  two  user  capabilities — a 
medical  single  sign  on  that  streamlines  the  login  process  to  allow  users  to 
sign  in  once  and  leverage  securely  stored  credentials  to  automatically 
access  the  other  available  applications,  and  a  context  management 
capability  that  will  automatically  present  a  patient’s  data  within  all 
applications.  Other  deliverables  include  enhanced  infrastructure  services, 
a  development  test  center/environment  configuration,  and  clinical  data 
repository  upgrades. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Defense  Health  Agency 
Program  owner:  PEO  DHMS 

Prime  contractor:  General  Dynamics  Information  Technology 
Total  number  of  contractors  (as  of  May  2016):  Not  applicable 


Table  13:  Integrated  Electronic  Health  Record  Increment  1  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

Feb  2013 

Latest 
estimate 
March  2015 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$69.8 

$46.7 

Procurement 

122.8 

71.2 

Acquisition  operations  and  maintenance 

173.7 

131.4 

Total  acquisition  cost 

$366.3 

$249.3 

Life-cycle  cost  estimate 

Operations  and  support 

659.6 

58.3 

Total  life-cycle  cost  estimate 

$1,025.9 

$307.6 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  iEHR  Inc  1  program  was  in  the  sustainment  phase 
and,  according  to  a  2016  MAIS  Annual  Report,  the  program  is  closed  out 
as  an  acquisition  program. 

Decreased  planned  cost  estimate 

The  iEHR  Inc  1  program’s  March  2015  estimate  for  total  life-cycle  cost 
was  $307.6  million,  a  decrease  of  $718.3  million  (-70  percent)  from  the 
original  total  life-cycle  estimate  of  $1,025.9  million.  Likewise,  the 
program’s  total  estimated  acquisition  cost  was  $249.3  million,  a  decrease 
of  $1 17  million  (-32  percent)  from  the  original  total  acquisition  cost 
estimate  of  $366.3  million.  The  reduction  in  estimated  cost  was  primarily 
because  of  reductions  in  the  program’s  scope. 

Exceeded  planned  schedule  estimate 

The  iEHR  Inc  1  program’s  original  objective  schedule  estimate  for  full 
deployment  decision  was  December  2013.  When  compared  to  the  actual 
full  deployment  decision  date  of  November  2014  with  the  original  date, 
the  program’s  schedule  slipped  1 1  months.  The  program  achieved  full 
deployment  in  April  2015,  1  month  ahead  of  schedule. 

Met  all  of  the  system  performance  targets 

The  iEHR  Inc  1  program  had  met  all  three  of  the  performance  targets 
which  were  (1)  essential  business  function  1-context  management,  (2) 
single  sign  on,  and  (3)  roaming 
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Army — Integrated 
Personnel  and  Pay 
System — Army  Increment 
1  (IPPS-AInc  1) 


Program  description 

The  IPPS-A  Inc  1  program  is  expected  to  provide  a  24-hour,  web-based, 
integrated  human  resources  system  to  soldiers,  human  resource 
professionals,  combatant  commanders,  personnel  and  pay  managers, 
and  other  authorized  users.  Specifically,  the  system  is  expected  to 
provide  a  single,  multi-component,  trusted  database  with  a  single  record 
for  all  Army  soldiers,  and  serve  as  a  trusted  data  source  for  personnel 
and  human  resources  data  for  the  entire  Army.  Soldiers  will  have  web- 
based  access  to  their  personnel  data,  enabling  them  to  better  manage 
their  careers  and  ensure  accuracy  of  information.  The  system  will  allow 
for  interface  communications  and  new  multi-component  reports  that 
include  a  Soldier  Record  Brief  for  all  Army  components  (Active,  National 
Guard  and  Reserve). 


Program  essentials  (as  of  October  2016) 

DOD  component:  Department  of  the  Army 

Program  owner:  Department  of  the  Army,  Deputy  Chief  of  Staff,  G-1 
Prime  contractors:  EDC  Consulting,  LLC  -  System  Integration,  Booz  Allen 
Hamilton-Support  Services 

Total  number  of  contractors  (as  of  May  2016):  N/A:  Program  is  in 
sustainment 


Table  14:  Integrated  Personnel  and  Pay  System — Army  Increment  1  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
March  2012 

Latest 
estimate 
January  2015 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$154.8 

$190.9 

Procurement 

2.2 

.5 

Total  acquisition  cost 

$157.0 

$191.4 

Life-cycle  cost  estimate 

Operations  and  support 

201.4 

177.5 

Total  life-cycle  cost  estimate 

$358.4 

$368.9 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  IPPS-A  Inc  1  program  had  achieved  full 
deployment  and  was  in  the  sustainment  phase.  The  program  was  re¬ 
baselined  after  implementing  strategic  recommendations  by  an 
independent  review  team  established  in  June  2013  that  determined  the 
system  design  was  sufficient  to  meet  Increment  I  requirements.  The  re¬ 
baselined  acquisition  program  baseline  was  signed  by  the  Milestone 
Decision  Authority  in  February  2015. 

Exceeded  planned  cost  estimate 

As  of  January  2015,  the  IPPS-A  Inc  1  program’s  total  estimated  life-cycle 
cost  was  $368.9  million,  an  increase  of  $10.5  million  (3  percent)  from  the 
original  total  estimated  life-cycle  cost  of  $358.4  million.  The  total 
estimated  acquisition  cost  was  $191.4  million  (as  of  February  2015),  an 
increase  of  $34.4  million  (22  percent)  from  the  original  total  acquisition 
cost  of  $157  million.  According  to  program  officials,  the  increase  in  cost 
was  a  direct  result  of  deferring  continuity  of  operations  requirements  and 
capturing  the  testing  environment  requirement  under  Increment  2. 

Exceeded  planned  schedule  estimate 

The  IPPS-A  Inc  1  program  achieved  Milestone  C  and  Full  Deployment 
Decision  about  6  months  after  the  original  threshold  dates  in  August  2013 
and  October  2013,  respectively.  During  the  acquisition,  the  delay 
associated  with  the  slip  for  a  Milestone  C  decision  triggered  a  significant 
change  to  the  program’s  schedule.  The  program  has  achieved  full 
deployment  and  is  now  in  the  sustainment  phase. 

Met  all  of  the  system  performance  targets 

The  IPPS-A  Inc  1  program  had  met  all  three  of  the  performance  targets 
which  were  (1)  net  ready,  (2)  interoperability,  and  (3)  availability 
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Army — Integrated 
Personnel  and  Pay 
System — Army  Increment 
2  (IPPS-AInc  2) 


Program  description 

The  IPPS-A  Inc  2  program  is  expected  to  provide  a  24-hour,  web-based, 
integrated  Human  Resources  system  for  Army  personnel.  It  is  to  deliver 
an  integrated  personnel  and  pay  services  for  all  Army  components 
(Active,  National  Guard,  and  Reserve),  building  on  a  trusted  database 
delivered  by  the  IPPS-A  inc  1  program.  The  IPPS-A  Inc  2  program  is 
planned  to  improve  the  Army’s  financial  management  processes  in 
support  of  military  personnel  and  payroll  by  linking  the  personnel  and 
payroll  functions  for  all  Army  personnel,  eliminating  duplicate  data  entry, 
reducing  complex  system  maintenance,  and  minimizing  pay 
discrepancies.  The  IPPS-A  Inc  2  program  is  expected  to  account  for 
status  changes  between  Active,  Reserve,  and  National  Guard 
components  to  ensure  accurate  service  time,  minimize  impact  on 
individual  pay,  credit  for  service,  and  other  benefits. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Department  of  the  Army 

Program  owner:  Department  of  the  Army,  Deputy  Chief  of  Staff,  G-1 
Prime  contractor:  CACI-System  Integration,  Booz  Allen  Hamilton-Support 
Services 

Total  number  of  contractors  (as  of  May  2016):  38 


Table  15:  Integrated  Personnel  and  Pay  System — Army  Increment  2  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
March  2015 

Latest 
estimate 
March,  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$644.3 

$709.7 

Procurement 

81.5 

110.5 

Total  acquisition  cost 

$725.8 

$820.2 

Life-cycle  cost  estimate 

Operations  and  support 

1,025.8 

1,261.7 

Total  life-cycle  cost  estimate 

$1,751.0 

$2,081.9 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  IPPS-A  Inc  2  program  is  in  the  development  and 
deployment  phase  of  the  defense  acquisition  life  cycle.  A  Milestone  B 
decision  was  achieved  in  December  2014.  In  fiscal  year  2016,  the 
program  completed  several  reviews — system  requirements,  system 
functional,  and  integrated  baseline.  The  program  indicated  requirements 
growth  as  an  area  of  potential  risk. 

Exceeded  planned  cost  estimate 

As  of  March  2016,  the  IPPS-A  Inc  2  program’s  total  life-cycle  cost 
estimate  was  $2,081.9  million,  an  increase  of  $330.9  million  (19  percent) 
from  the  original  estimated  life-cycle  cost  of  $1 ,751 .0  million.  The  current 
total  estimated  acquisition  cost  estimate  was  $820.2  million,  an  increase 
of  $94.3  million  (13  percent)  from  the  original  estimated  acquisition  cost  of 
$725.8  million. 

Met  planned  schedule  estimate 

The  IPPS-A  Inc  2  program  achieved  Milestone  B  on  19  December  2014, 
within  the  planned  threshold  schedule. 

Not  yet  applicable 

The  IPPS-A  Inc  2  program  was  still  early  in  development  and  system 
performance  data  was  not  available. 
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Air  Force — Integrated 
Strategic  Planning  and 
Analysis  Network 
Increment  4  (ISPAN  Inc  4) 


Program  description 

The  ISPAN  Inc  4  program  is  expected  to  be  a  system-of-systems  that 
spans  multiple  security  enclaves  for  strategic  and  operational  level 
planning  and  leadership  decision  making.  The  system  is  composed  of  two 
primary  components:  (1)  a  global  adaptive  planning  collaborative 
information  environment  that  manages  strategy-to-execution  planning 
across  all  United  States  Strategic  Command  (USSTRATCOM)  mission 
areas  and  (2)  a  mission  planning  and  analysis  system  that  supports 
developing  nuclear  and  conventional  force  application  plans.  Both 
components  establish  a  framework  to  support  the  USSTRATCOM’s 
effects-based  planning  and  analysis  activities. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Air  Force 
Program  owner:  USSTRATCOM 

Prime  contractors:  BAE,  Leidos,  Lockheed  Martin,  and  Northrop 
Grumman 

Total  number  of  contractors  (as  of  May  2016):  19 


Table  16:  Integrated  Strategic  Planning  and  Analysis  Network  Increment  4  Program 
costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
August  2014 

Latest 
estimate 
June  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$149.3 

$149.3 

Acquisition  O&M 

$13.2 

$13.2 

Total  Acquisition  Cost 

$162.5 

$162.5 

Life  cycle  cost  estimate 

Operations  and  support 

$24.7 

$24.7 

Total  life  cycle  cost  estimate 

$187.2 

$187.2 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  June  201 6,  the  ISPAN  Inc  4  program  was  in  the  engineering  and 
manufacturing  development  phase  of  the  Defense  Acquisition 
Management  System.  According  to  officials,  the  program  was  on  track  to 
remain  within  the  schedule,  cost,  and  performance  thresholds  identified  in 
the  original  estimate,  with  no  major  risks.  The  program  has  not  been 
rebaselined. 

Met  planned  cost  estimate 

The  ISPAN  Inc  4  program’s  current  cost  estimates  have  not  changed 
since  their  first  estimate. 

Met  planned  schedule  estimate 

The  ISPAN  Inc  4  program’s  current  schedule  estimates  have  not  changed 
since  their  first  estimate 

Not  yet  applicable 

The  ISPAN  Inc  4  program  was  still  early  in  development  and  system 
performance  data  was  not  yet  available. 
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Air  Force — Joint  Space 
Operations  Center, 

Mission  System  Increment 
2  (JMS  Inc  2) 


Program  description 

The  JMS  Inc  2  Program  is  expected  to  replace  the  legacy  Space  Defense 
Operations  Center  hardware  and  services  with  extensible  and  sustainable 
infrastructure.  The  effort  is  planned  to  integrate  components  of  the  Space 
Situational  Awareness  mission  applications  and  Command  and  Control 
capabilities  into  the  Joint  Space  Operations  Center  to  create  timely, 
actionable  knowledge  necessary  for  maintaining  space  superiority  and 
exercising  command  and  control  capabilities  of  space  forces. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Air  Force 

Program  owner:  Air  Force  Space  Command 

Prime  contractor:  N/A,  managed  by  federal  government 

Total  number  of  contractors  (as  of  May  2016):  7 


Table  17:  Joint  Space  Operations  Center,  Mission  System  Increment  2  Program 
costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

June  2013 

Latest 
estimate 
March  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$312.7 

$462.8 

Total  acquisition  cost 

$312.7 

$462.8 

Life-cycle  cost  estimate 

Operations  and  sustainment 

787.8 

503.1 

Total  life-cycle  cost  estimate 

$1,100.5 

$965.9 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  March  2016,  the  JMS  Inc  2  program  was  in  the  engineering  and 
manufacturing  development  acquisition  phase.  Due  to  issues  associated 
with  schedule  delays  of  more  than  12  months  and  acquisition 
development  costs  increasing  more  than  25  percent,  the  program 
underwent  a  critical  change.  A  critical  change  report  was  delivered  to 
Congress  in  September  2016  due  to  the  delayed  completion  of  the 
program’s  Milestone  C  and  full  deployment  decision  by  more  than  12 
months.  According  to  the  report,  three  factors  that  caused  a  critical 
change  were  (1)  the  program’s  original  schedule  was  aggressive  and 
contained  a  significant  level  of  concurrency,  (2)  funding  and  manpower 
issues,  and  (3)  contracting  issues. 

Decreased  planned  cost  estimate 

As  of  March  2016,  the  JMS  Inc  2  program’s  total  life-cycle  cost  estimate 
was  $965.9  million,  a  decrease  of  $134.6  million  (-12  percent)  from  the 
original  total  estimate  of  $1,100.5  million.  The  latest  estimate  for  total 
estimated  acquisition  cost  was  $462.8  million,  an  increase  of  $150.1 
million  (48  percent)  from  the  original  acquisition  cost  estimate  of  $312.7 
million.  The  decrease  for  the  total  life-cycle  cost  estimate  was  because 
the  original  estimate  for  operations  and  sustainment  costs  was  20  years 
after  fielding  while  the  new  estimate  is  for  10  years  after  fielding. 

Exceeded  planned  schedule  estimate 

Due  to  significant  delays  in  the  JMS  Inc  2  program’s  aggressive  schedule, 
the  program  underwent  a  rebaseline  effort  and  issued  a  critical  change 
report  in  September  2016.  The  new  estimate  for  a  full  deployment 
decision  is  May  201 9.  With  an  original  schedule  date  of  July  201 6,  this 
represents  a  2-year,  10-month  slippage. 

Not  yet  applicable 

The  JMS  Inc  2  program  was  still  early  in  development  and  system 
performance  data  was  not  yet  available. 
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Army — Logistics 
Modernization  Program 
Increment  2  (LMP  Inc  2) 


Program  description 

The  LMP  Inc  2  program  is  expected  to  deliver  an  enterprise  solution  that 
builds,  sustains,  and  generates  warfighting  capabilities  using  a  fully- 
integrated  supply  chain  and  maintenance,  repair,  and  overhaul  system. 
The  LMP  Inc  2  program’s  support  is  critical  to  the  Army’s  ability  to 
achieve  an  integrated  system  that  enables  materiel  readiness  and  asset 
management  and  accountability,  architecture,  and  acquisition  compliancy. 
When  deployed,  it  is  expected  to  deliver  capabilities  that  address  critical 
Army  Materiel  Command  requirements  for  automation  of  the  industrial 
base  shop  floor  operations,  standardization  of  ammunition  automatic 
identification  technology,  non-Army  managed  items  management,  Army 
prepositioned  stock  planning,  national  maintenance  program,  and  the 
expansion  and  refinement  of  the  integration  between  Army  enterprise 
resource  planning  systems. 


Program  essentials  (as  of  October  2016) 

DOD  component:  Army 

Program  owner:  PEO  EIS 

Prime  contractor:  CSRA,  LLC 

Total  number  of  contractors  (as  of  May  2016):  188 


Table  18:  Logistics  Modernization  Program  Increment  2  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 

Aug  2013 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Army  working  capital  funds-capital 
investment  program 

$344.7 

$370.8 

Army  working  capital  funds-operations 

60.4 

48.4 

Total  acquisition  cost 

$405.1 

$419.2 

Life-cycle  cost  estimate 

Operations  and  support 

$324.8 

$317.6 

Total  life-cycle  cost  estimate 

$729.9 

$736.8 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  June  2016,  the  LMP  Inc  2  program  was  in  the  operation  and 
support  phase  of  the  acquisition  life  cycle.  The  program  went  to  full 
deployment  in  September  2016.  The  program  will  continue  to  provide 
national  level  logistics  and  financial  operations  to  Army  arsenals  and 
depots. 

Exceeded  planned  cost  estimate 

As  of  July  2016,  the  LMP  Inc  2  program’s  total  estimated  life-cycle  cost 
estimate  was  $736  million,  an  increase  of  $6.9  million  (1  percent)  from  the 
original  total  life-cycle  cost  estimate  of  $729.9  million.  Likewise,  the  total 
current  estimated  acquisition  cost  estimate  was  $419.2  million,  an 
increase  of  $14.1  million  (3  percent)  from  the  original  estimate  of  $405.1 
million.  However,  these  increases  were  within  the  threshold  estimate 
established  for  the  program. 

Exceeded  planned  schedule  estimate 

The  LMP  Inc  2  program  had  a  schedule  slip  of  5  months  when  comparing 
the  original  objective  full  deployment  decision  date  of  October  2015  to  the 
current  estimate  of  July  2016.  However,  the  schedule  slip  was  within  the 
established  threshold  date  of  April  2016. 

Met  all  of  the  system  performance  targets 

The  LMP  Inc  2  program  had  met  all  seven  of  the  system  performance 
targets  which  were  (1)  support  net-centric  military  operations,  (2)  enter 
and  be  managed  in  the  network,  (3)  net  readiness  exchange  information, 
(4)  survivability  system  information  assurance,  (5)  survivability  information 
assurance  system  disaster  recovery,  (6)  sustainment  operational 
availability,  and  (7)  training  proficiency  level. 
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Defense  Information 
Systems  Agency — 
Teleport  Generation  3 
(Teleport  Gen  3) 


Program  description 

The  Teleport  Gen  3  program  is  expected  to  be  a  satellite  communications 
gateway  that  will  enable  the  transportation  of  voice,  video,  and  data 
information  to  deployed  forces,  and  links  the  deployed  warfighter  to  the 
sustaining  base.  The  system  is  expected  to  provide  expanded  system 
capacity,  throughput,  and  functional  capabilities  to  greatly  enhance 
support  to  tactical  and  deployed  warfighters  worldwide. 

Program  essentials  (as  of  October  2016) 

DOD  component:  Defense  Information  Systems  Agency 
Program  owner:  DISA  Component  Acquisition  Executive 
Prime  contractors:  Booz  Allen  Hamilton,  Femme  Comp  Inc.,  and  Itility 
Total  number  of  contractors  (as  of  May  2016):  5 


Table  19:  Teleport  Generation  3  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
September  2010 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$19.20 

$18.30 

Procurement 

243.30 

229.60 

Total  acquisition  cost 

$262.50 

$260.30 

Life-cycle  cost  estimate 

Operations  and  support 

301.20 

315.00 

Total  life-cycle  cost  estimate 

$563.70 

$575.30 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  July  2016,  the  Teleport  Gen  3  program  was  in  the  production  and 
deployment  acquisition  phase. 

Exceeded  planned  cost  estimate 

As  of  July  2016,  the  Teleport  Gen  3  program’s  current  total  life-cycle  cost 
estimate  was  $575.3  million,  an  increase  of  $1 1 .6  million  (2  percent)  from 
the  original  estimate  of  $563.7  million. 

Exceeded  planned  schedule  estimate 

The  Teleport  Gen  3  program  had  a  schedule  slip  of  3  years  and  2 
months,  from  January  2013  to  March  2016  when  comparing  the  original 
baseline  Milestone  C  (Gen  3,  Phase  3)  objective  schedule  estimate. 

Did  not  fully  meet  all  of  the  system  performance  targets 

The  Teleport  Gen  3  program’s  Phase  1  performance  target  had  been 
met.  However,  performance  targets  for  Phases  2  and  3  had  not  been  met. 
According  to  the  program  office,  all  developmental  and  operational  testing 
had  been  completed  for  Teleport  Gen  3  Phases  1  and  2.  Additional 
testing  is  planned  for  Phase  3. 
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Program  description 

The  TMC  program  is  a  suite  of  systems  that  is  expected  to  provide 
mission  command  computing  capabilities  to  Army  commanders  and  their 
staffs,  consisting  of  a  user-customizable  common  operating  picture 
enabled  with  real-time  collaboration.  These  capabilities,  together  with 
voice-over-internet-protocol,  will  provide  real-time  situational  awareness 
and  decision  support  across  the  formation  (battalion  thru  Army  Service 
Component  Command).  A  component  of  the  mission  command  capability 
is  the  ability  to  layer  information  from  multiple  functions  on  the  same  map 
display  and  share  among  command  posts,  allowing  all  parties  to  have  a 
common  view.3 

Program  essentials  (as  of  October  2016) 

DOD  component:  Army 
Program  owner:  Mission  Command 

Prime  contractor:  General  Dynamics  (previous);  currently  a  government 
led  effort  by  Army  Research  and  Development  Engineering  Center 
Total  number  of  contractors  (as  of  May  2016):  N/A.  Army  is  the  lead 


Table  20:  Tactical  Mission  Command  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
July  2005 

Latest 
estimate 
March  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$319.5 

$439.3 

Procurement 

500.9 

1,752.8 

Total  acquisition  cost 

$820.4 

$2,192.1 

Life-cycle  cost  estimate 

Operations  and  support 

73.4 

123.4 

Total  life-cycle  cost  estimate 

$893.8 

$2,315.5 

Source:  DOD  data  |  GAO-17-322 


3While  these  capabilities  are  currently  provided  by  Command  Post  of  the  Future,  the 
Tactical  Mission  Command  system  is  transitioning  to  the  web-based  Command  Post 
Computing  Environment. 
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Program  status 

As  of  March  2016,  the  TMC  program  is  currently  post  Milestone  C.  The 
program  has  been  rebaselined  two  times.  A  critical  change  report  was 
issued  in  response  to  a  cost  breach  caused  by  a  growth  in  the  program’s 
scope  growth  that  was  considered  a  “fact  of  life”  change.  As  of  March 
2016,  program  officials  reported  they  are  on  track  to  remain  within  the 
schedule,  cost,  and  performance  thresholds  identified  in  the  original 
estimate. 

Exceeded  planned  cost  estimate 

As  of  March  2016,  the  TMC  program’s  total  life  cycle  cost  estimate  was 
$2,315.50  million,  an  increase  of  $1,421.7  million  (159  percent)  from  the 
original  total  life  cycle  cost  estimate  of  $893.80  million.  The  TMC 
program’s  latest  total  acquisition  cost  estimate  was  $2,192.1  million,  an 
increase  of  $1,371.7  million  (167  percent)  from  the  original  acquisition 
estimate  of  $820.40  million  in  March  201 6. 4  We  reported  in  March  2016 
that  the  Army’s  TMC  program  estimated  program  development  costs  had 
increased  by  45  percent  over  the  original  estimate  due  to  program  scope 
changes  derived  from  the  realignment  of  Command  Post  of  the  Future  as 
a  foundation  for  Mission  Command  Collapse,  the  integration  of 
Personalized  Assistant  that  Learns,  and  the  incorporation  of  future  force 
requirements.  According  to  a  2016  MAIS  Annual  Report,  the  program  was 
on  track  to  remain  within  the  schedule,  cost,  and  performance  thresholds 
identified  in  the  original  estimate. 

Met  planned  schedule  estimate 

As  we  reported  in  our  March  2016  report,  the  TMC  program  experienced 
a  slight  slippage  in  schedule  (3  months),  however,  it  was  within  the 
established  threshold  date.  Program  officials  report  they  are  still  on  track 
to  reach  full  deployment  by  December  2018. 

Met  all  of  the  system  performance  targets 

As  of  March  2016,  the  TMC  program  had  met  all  three  of  the  performance 
targets  which  were  (1)  net  ready,  (2)  display  the  common  operational 
picture,  and  (3)  disseminate  (create  and  exchange)  orders. 


4GAO-1 6-336. 
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DOD — Theater  Medical 
Information  Program — 
Joint  Increment  2 


(TMIP-J  Inc  2) 


Program  description 

The  TMIP-J  Inc  2  program  is  expected  to  integrate  components  of  the 
Military  Health  System  sustaining  base  systems  and  the  Services' 
medical  information  systems  to  ensure  timely  interoperable  medical 
support  for  mobilization,  deployment,  and  sustainment  of  all  theater  and 
deployed  forces.  The  system  is  expected  to  enhance  clinical  care  and 
information  capture  at  all  levels  of  care  in  theater,  transmits  critical 
information  to  the  theater  commander;  the  evacuation  chain  for  combat 
and  non-combat  casualties;  and  forges  the  theater  links  of  the  health 
record  to  the  sustaining  base  and  the  Department  of  Veterans  Affairs. 
The  TMIP-J  program  is  the  medical  component  of  the  Global  Combat 
Support  System. 

Program  essentials  (as  of  October  2016) 

DOD  components:  DOD,  Joint  Participants:  U.S.  Army;  U.S.  Navy  and 
Marine  Corps;  U.S.  Air  Force 

Program  owner:  PEO  Defense  Healthcare  Management  Systems 
Prime  contractor:  KRATOS  Inc. 

Total  number  of  contractors  (as  of  April  2016):  6 


Table  21 :  Theater  Medical  Information  Program — Joint  Increment  2  Program  costs 

(then-year  dollars  in  millions) 

First  acquisition 
program  baseline 
December  2007 

Latest 
estimate 
July  2016 

Acquisition  cost  estimate 

Research,  development,  test,  and 
evaluation 

$124.2 

$323.7 

Procurement 

8.6 

16.7 

Operations  and  maintenance 

100.5 

267.8 

Total  acquisition  cost 

$233.3 

$608.2 

Life-cycle  cost  estimate 

Operations  and  support 

442.8 

448.2 

Total  life-cycle  cost  estimate 

$676.1 

$1,056.4 

Source:  DOD  data  |  GAO-17-322 
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Program  status 

As  of  July  2016,  the  TMIP-J  Inc  2  program  was  reported  as  fully  deployed 
and  was  in  the  sustainment  phase.  The  system  consists  of  two 
increments — Increment  1  was  fielded  in  2003,  and  development  of 
Increment  2  was  being  done  in  three  releases.  Release  1  was  fielded  in 
2009  and  Release  2  was  fielded  in  2013.  Release  3,  is  the  last  major 
release.  In  February  2016,  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  declared  that  the  TMIP-J  Inc  2 
program  had  met  its  full  deployment  decision  action  items.  He  delegated 
Milestone  Decision  Authority  to  the  Program  Executive  Officer,  Defense 
Healthcare  Management  Systems,  and  authorized  the  Services  to  declare 
full  deployment  in  accordance  with  their  Service  acquisition  strategies. 

Exceeded  planned  cost  estimate 

As  of  July  2016,  the  TMIP-J  Inc  2  program’s  total  life  cycle  cost  estimate 
was  $1,056.4  million,  an  increase  of  $380.3  million  (56  percent)  from  the 
original  estimate  of  $676.1  million.  The  total  acquisition  cost  estimate  was 
$608.2  million,  an  increase  of  $374.9  million  (161  percent)  from  the 
original  estimate  of  $233.3  million. 

Exceeded  planned  schedule  estimate 

The  TMIP-J  Inc  2  program  had  a  5-year,  6-month  change  to  the  full 
deployment  decision  date  when  a  breach  occurred.  Program  officials 
stated  the  schedule  change  was  due  to  a  mandate  to  increase  the  scope 
of  the  program.  In  turn,  a  revised  acquisition  program  baseline  was 
approved  in  January  2012.  The  objective  date  for  full  deployment 
changed  from  June  2008  to  December  2013,  and  a  threshold  date 
changed  from  December  2008  to  June  2014.  The  TMIP-J  Inc  2  program 
reached  full  deployment  in  May  2016  and  has  transitioned  into  the 
sustainment  phase. 

Met  all  of  the  system  performance  targets 

The  TMIP-J  Inc  2  program  had  met  all  ten  of  the  performance  targets 
which  were  (1)  net  ready,  (2)  data  transfer  data  availability,  (3)  data 
availability,  currency,  and  responsiveness — medical  logistics  visibility,  (4) 
data  availability,  currency  and  responsiveness — patient  visibility,  (5)  data 
availability,  currency  and  responsiveness — medical  infrastructure 
readiness,  (6)  data  availability,  currency  and  responsiveness — individual 
medical  readiness,  (7)  data  availability,  currency  and  responsiveness — 
illness  and  injury  rates,  (8)  data  availability,  currency  and 
responsiveness — reportable  medical  events,  (9)  standards  compliance, 
and  (10)  system  operational  availability  and  responsiveness. 
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ACQUISITION 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


MAR  2  3  2017 


Ms.  Carol  C.  Harris 

Director,  Information  Technology  Acquisition  Management  Issues 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington  DC  20548 

Dear  Ms.  Harris, 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report 
GAO- 17-322,  “DOD  MAJOR  AUTOMATED  INFORMATION  SYSTEMS: 


Improvements  Can  Be  Made  in  Applying  Leading  Practices  for  Managing  Risk  and 
Testing,"  dated  February  24,  2017  (GAO  Code  100780).  Responses  to  the  report 


recommendations  are  attached. 


Perftrmmg  the  Duties  of  the 
Assistant  Secretary  of  Defense 
for  Acquisition 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  FEBRUARY  24,  2017 
GAO-17-322  (GAO  CODE  100780) 

“DOD  MAJOR  AUTOMATED  INFORMATION  SYSTEMS:  IMPROVEMENTS  CAN 
BE  MADE  IN  APPLYING  LEADING  PRACTICES  FOR  MANAGING  RISK  AND 

TESTING” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  1:  To  help  improve  the  management  of  DOD's  MAIS  programs,  GAO 
is  recommending  that  the  Secretary  of  Defense  take  the  following  action:  direct  the  Secretary  of 
the  Army  to  direct  the  Product  Manager  for  Global  Combat  Service  Support  -  Army  (GCSS- 
Army)  Increment  1  to  establish  standard  operating  procedures  for  managing  risks  that  include 
guidance  for  establishing  thresholds  and  bounds  for  key  risk  areas. 

DoD  RESPONSE:  Concur.  GCSS-Army  Increment  1  is  already  using  an  established  standard 
operating  procedure  for  managing  risk  which  includes  the  concepts  of ‘'thresholds”  and 
“bounds.”  As  a  DoD  organization,  GCSS-Army  adheres  to  the  standards  articulated  in  the  DoD 
Risk,  Issue,  and  Opportunity  Management  Guide  (RIO).  The  leading  practices  identified  on 
Table  3  of  the  report  (attributed  to  the  Software  Engineering  Institute’s  Capability  Maturity 
Model  for  Acquisition  (CMMI-ACQ))  are  parallel  to  the  identified  practices  in  the  RIO.  While 
there  is  minor  variation  in  terminology — CMMI-ACQ  is  focused  commercially  and  DoD  RIO  is 
focused  on  Defense  acquisition — the  intent  to  manage  risk  through  a  framework  of  thresholds 
and  bounds  is  consistent  across  both  models.  GCSS-Army  is  successfully  applying  the  leading 
practices  as  part  of  its  documented  risk  management  process. 

RECOMMENDATION  2:  To  help  improve  the  management  of  DOD's  MAIS  programs,  GAO 
is  recommending  that  the  Secretary  of  Defense  take  the  following  action:  direct  the  Secretary  of 
the  Air  Force  to  direct  the  program  manager  for  AOC-WS  Inc  10.2  to  develop  an  overall  risk 
mitigation  plan  to  guide  the  implementation  of  individual  risk  mitigation  and  contingency  plan 
activities. 

DoD  RESPONSE:  Concur.  While  the  AOC  WS  System  Program  Office  (SPO)  has  a  draft  Risk 
and  Opportunities  Management  (ROM)  Plan  it  has  neither  been  published  nor  rigorously 
adhered  to.  One  of  the  SPO’s  lessons  learned  and  process  improvement  initiatives  coming  out  of 
the  AOC  10.2  Critical  Change  was  to  revamp  ROM  practices  in  the  SPO.  The  Program  Manager 
has  taken  steps  to  address  this  need  including  re-initiating  weekly  ROM  meetings,  both  internally 
and  jointly,  with  the  prime  contractor  and  reviewing  program  risks  bi-weekly  with  the  Program 
Executive  Officer.  The  ROM  Plan  will  be  published  by  3  Apr  17. 
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RECOMMENDATION  3:  To  help  improve  the  management  of  DOD's  MAIS  programs,  GAO 
is  recommending  that  the  Secretary  of  Defense  take  the  following  action:  direct  the  Secretary  of 
the  Air  Force  to  direct  the  program  manager  for  JMS  to  appoint  a  chief  developmental  tester  to 
oversee  systems  testing  and  integration  activities. 

DoD  RESPONSE:  Concur.  The  JMS  program  has  identified  and  selected  a  qualified  individual 
to  fill  the  Chief  Developmental  Test  position.  The  new  Developmental  Test  Chief  will  be  in 
place  in  July  2017. 
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GAO  Contact 

Carol  C.  Harris  at  (202)  512-4456  or  harrisc@gao.gov 

In  addition  to  the  contact  name  above,  the  following  staff  also  made  key 
contributions  to  this  report:  Eric  Winter  (Assistant  Director),  John  Ortiz 
(Analyst  in  Charge),  Neha  Bhatt,  Nancy  Glover,  Kate  Nielsen,  and  Teresa 
Smith. 


Staff 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and  investigative 
arm  of  Congress,  exists  to  support  Congress  in  meeting  its  constitutional 
responsibilities  and  to  help  improve  the  performance  and  accountability  of  the 
federal  government  for  the  American  people.  GAO  examines  the  use  of  public 
funds;  evaluates  federal  programs  and  policies;  and  provides  analyses, 
recommendations,  and  other  assistance  to  help  Congress  make  informed 
oversight,  policy,  and  funding  decisions.  GAO’s  commitment  to  good  government 
is  reflected  in  its  core  values  of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  GAO’s  website  (http://www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  website  newly  released  reports,  testimony,  and  correspondence.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products,  go  to  http://www.gao.gov 
and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of  production  and 
distribution  and  depends  on  the  number  of  pages  in  the  publication  and  whether 
the  publication  is  printed  in  color  or  black  and  white.  Pricing  and  ordering 
information  is  posted  on  GAO’s  website,  http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card,  MasterCard, 

Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Linkedln,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 

Visit  GAO  on  the  web  at  www.gao.gov  and  read  The  Watchblog. 

To  Report  Fraud, 

Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512-4400, 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125, 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 

Washington,  DC  20548 

Strategic  Planning  and 
External  Liaison 

James-Christian  Blockwood,  Managing  Director,  spel@gao.gov,  (202)  512-4707 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7814, 
Washington,  DC  20548 
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